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ABSTRACT 



This technical report provides an updated assessment of the 
problem of optimizing the periodicity of the Schools and Staffing Survey 
(SASS) . Making use of data from three rounds of SASS data collection (for 
school years 1987-88, 1990-91, and 1993-94) , the report extends and updates 
the preliminary findings and interim assessments that were previously 
published in a paper for the American Statistical Association (D. Ghosh et 
al . , 1994) and a technical report for the National Center for Education 
Statistics (NCES) (D. Ghosh et al . , 1995) . Part I of the report (Sections 1 
through 6) presents the analysis and findings, while Part II provides 
documentation and numerical results in terms of error projections for 
selected models, policy variables, cost rations, and periodicities. Basic 
results are provided for four probable error models. Section 1 provides an 
overview of the report and characterizes the general task of optimizing the 
periodicity of the SASS as one of identifying decision rules that will result 
in the selection of intersurvey time intervals and survey sample sizes that 
are in some sense "jointly optimal subject to a known, fixed set of cost 
constraints." While the first three rounds of the SASS were conducted at 
3 -year intervals, budget constraints meant that the next round was delayed to 
5 years later. Section 2 covers the main modeling and analysis features of 
this study, while section 3 provides a brief review of time series 
methodology in relation to the periodicity problem. Section 4 sets forth 
summary conclusions and recommendations, concluding with the suggestion that 
the NCES should consider adopting an alternating large and small sample 
design for the SASS with an appropriate full -sample periodicity (such as 6 
years) together with a mid-period fractional-sample data collection at the 
halfway time between full sample data collections. Section 6 provides an 
extensive bibliography for the whole report. Part II has four sets of 
detailed appendix tables. Appendix A provides large-sample scenarios for 
public schools and four states; Appendix B provides an alternating large and 
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small sample design. Appendix C provides similar scenarios for private 
schools, and Appendix D provides an alternating large and small sample design 
for private schools. (Contains 2 figures, 8 tables, and 54 references.) (SLD) 
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The Working Paper Series was created in order to preserve the information contained 
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Consequently, we encourage users of the series to consult the individual authors for citations. 

To receive information about submitting manuscripts or obtaining copies of the series, 
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Executive Summary 



This technical report provides an updated assessment of the problem of optimizing the periodicity 
of the Schools and Staffing Survey (SASS). Making use of data from three rounds of SASS data 
collection (for school years 1987-88, 1990-91, and 1993-94), the report extends and updates the 
preliminary findings and interim assessments that were set forth in an American Statistical 
Association proceedings paper [Ghosh et ai, 1994] and in a previous technical report prepared for 
the National Center for Education Statistics [Ghosh et ai, 1995], 

Part I of this report presents the analysis and findings, while Part II provides documentation and 
numerical results in terms of error projections for selected models, policy variables, cost ratios and 
periodicities. Basic results are provided for four “probable error” models, designated as Models 1, 
2A, 3A, and 4M. Model 1 was developed in a previous study and Models 2A and 3 A were derived 
from similar prior models. Model 4M combines elements from two prior models. Additional 
results are provided using two of the four models, namely Models 3A and 4M, that behave in a 
consistent but not identical manner over various scenarios and designs. 

Section 1 provides an overview of the report and characterizes the general task of “optimizing the 
periodicity of SASS” as one of identifying decision rules that will result in the selection of 
intersurvey time intervals and survey sample sizes that are in some sense “jointly optimal subject to 
a known, fixed set of cost constraints.” While SASS was conducted at three-year intervals for the 
first three rounds of the survey, budget constraints have led to a delay of the next round of data 
collection to school year 1998-99, which is five years after the last round in 1993-94. NCES has 
asked the research team for the present study to consider the effect of periodicities of 4, 5, or 6 
years and variable-to- fixed cost ratios of 30:70, 40:60, 50:50, 60:40, and 70:30 under a fixed total 
cost constraint roughly equivalent to the total cost of the 1993-94 SASS round. Section 1 also 
provides an overview of the concepts of direct survey costs (fixed and variable) as well as the 
imputed costs of “not knowing” - that is, the penalty (whether or not reduced to dollar terms) that 
a data user may incur as a result of using out-of-date data. Section 1 concludes with a brief review 
of the first three rounds of SASS. 

Section 2 covers the main modeling and analysis features of the present study, which has focused 
on the use of probable error models as decision support tools in the optimal choice of survey 
periodicities. Cost models with fixed and variable costs are described in relation to the assumption 
of fixed total (annualized) resources. Different choices of periodicities and other design features 
will then spend these resources in different ways. There is a detailed discussion of the extensions 
to the family of probable error models that had been developed in an earlier study. Statistical 
formulas are provided to generate the year-to-year evolution of the projected absolute errors under 
each of the four models and the cumulative, multi-year average effect for each model. The 
projected absolute errors are a combination of the sampling error (sampling absolute error) at each 
survey data collection time and the error resulting from the unobserved changes in the underlying 
processes (changes in school and teacher variables) between survey data collections. The modeling 
results were generated by ^plying SASS data for a selection of policy variables (seven for public 
schools, four for private schools, and one for public and private schools combined). 

There is a detailed review of the numerical results for large-sample scenarios, with emphasis on 
Models 3A and 4M, for both public schools and private schools. There are some fifteen scenarios 
in all, resulting from pairwise combinations of five possible variable-to-fixed cost ratios and of 
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three different periodicities (four, five, or six years). The effect of different cost ratio assumptions 
on the projected errors was not large, with average relative projected absolute errors varying only 
plus or minus 10 per cent from reference values at a 50:50 cost ratio to values for 30:70 and 70:30 
cost ratios. Tables and graphs for public schools included in section 2 highlight the conclusion that 
shorter periodicities (even though they involve smaller sample sizes) are uniformly better than 
longer periodicties at the national level. For State level public school data, longer periodicities 
(which have larger sample sizes) are somewhat preferable, based on the results for California, 

Iowa, and New York. 

As an alternative to the “fixed design” of the large-sample scenarios; that is, a fixed set of sample 
sizes repeated at fixed intervals of four, five, or six years, a novel alternating large-and-small- 
sample design was formulated and tested. The specific alternating design takes the sample sizes 
that would otherwise have been used in the fixed design with a five-year periodicity and carries out 
a large-sample data collection every six years. It then uses the resulting resource savings to 
conduct a fractional-sample (specifically, a one-fifth size sample) data collection three years after 
each of the large data collections. The results of the Alternating Design compared to the Fixed 
Design are discussed further in section 4. Section 2 concludes with a discussion of the uses and 
limitations of currently available probable-error modeling methods. 

Section 3 of the report provides a brief review of time series methods in relation to the periodicity 
problem. The early work of Scott, Smith, and Jones in the 1970s on time series methods for the 
analysis of repeated surveys is reviewed briefly. The relations between exponential smoothing 
models (applicable to random walk processes) and Kalman filter or state space approaches are 
outlined as a basis for a discussion of W. S mi th’s 1980 development of an equivalent sample size 
form of the Kalman filter and its application to random walk models of repeated surveys. Section 
3 reports on an inconclusive attempt to apply methods for replicated autoregressive processes to 
assess the possible dependence of components observed in the SASS periodic series , but 
unobserved in NCES’s Common Core of Data (CCD) series, on the observed components in the 
annual CCD series. A theoretical formulation was made, but available test data sets were not 
adequate to the planned analyses. Section 3 concludes with an analysis of the uses and limitations 
of time series modeling approaches. 

Section 4 sets forth summary conclusions and recommendations. A discussion of lessons from 
probable error modeling and time series approaches reinforces a number of findings described in 
section 2. Projected absolute errors are relatively insensitive to cost ratios over the specified range 
of variable-to-fixed cost ratios (from 30:70 to 70:30). Shorter periodicities, even though they 
involve smaller sample sizes, are better in terms of lower projected errors for national estimates. 

For State-level estimates (public schools only), longer periodicities are slightly better. 

Section 4 concludes with a recommendation “. . .that the National Center for Education Statistics 
should consider adopting an Alternating Large-and-Small-Sample Design for SASS with an 
appropriate full-sample periodicity (such as six years) together with a mid-period fractional-sample 
data collection to be conducted half way between the full-sample data collections. The mid-period 
data collection would provide a periodic update at the national level and for the larger States.” 

Section 5 is a technical supplement providing supplementary information which further describes 
the theoretical and empirical basis for some of the topics and analyses presented in earlier sections 
of the report Theoretical detail is provided on the equivalent sample size form of the Kalman filter 
and it application to random walk models of repeated surveys. This supplements the description in 
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section 3.2. A formulation of a Markovian state-space model to support the composite modeling 
and estimation of policy variables through the joint use of CCD and SASS data is provided. This 
supplements the description in section 3.3. 

Section 6 provides an extensive bibliography for the report as a whole. 

Part II contains four sets of detailed appendix tables of projected absolute errors for public and 
private schools and for large-sample scenarios and for the alternating large-and-small-sample 
designs. Appendix A provides large-sample scenarios for public schools at the national level 
(U.S.) and for the States of California, Iowa, and New York. Appendix B provides an alternating 
large-and-small-sample design for public schools at the same geogrt^hic levels. Appendix C 
provides large-sample scenarios for private schools (national level only) aiKl Appendix D provides 
an alternating large-and-small-sample design for private schools (national level only). 

* * * 

This report was prepared at Synectics for Management Decisions by Wray Smith, Dhiren Ghosh, 
and Michael Chang, with the assistance of S. Bhalla, M. Hu, and M. Saba. 

* * * 

KEY WORDS: Absolute error modeling; Data deterioration; Fixed and variable costs; 
Optimization; Repeated surveys; Time series modeling 
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Part I - Analysis and Findings 



1. Introduction and Outline of Research Problems Addressed 



1.1 Overview 

This report updates an interim assessment presented in a proceedings paper (Ghosh et al„ 1994) 
and in a technical report (Ghosh et al, 1995) of our previous study of methods and criteria in 
support of qjtimizing the periodicity of the Schools and Staffing Survey (SASS). The term 
“optimizing the periodicity of SASS” is intended to include any procedures that will result in 
selecting an intersurvey time interval and a survey sample size that are in some sense jointly 
optimal subject to a known, fixed set of cost constraints. 

The National Center for Education Statistics (NCES) has an ongoing agreement with the Bureau of 
the Census under which the bureau carries out the periodic collection of SASS data. There have 
now been three cycles of SASS data collection, spaced at three-year intervals, and a fourth cycle is 
now tentatively scheduled to take place five years after the third cycle. The programmatic 
justification for the ongoing cycle of data collections that make up the SASS survey system lies in 
the fact that SASS collects jointly analyzable school-level data on a variety of variables that are 
relevant to educational research and policy analysis. Such variables have to do with school and 
teacher (and pupil) characteristics and provide detail that is not available ftom the annual data on 
public schools compiled through the Common Core (rf Data (CCD) system. Among the detailed 
measures provided by SASS are estimates of teacher qualifications and turnover by subject area, 
teacher and administrator compensation, and ethnic composition of students as well as faculty. In 
the case of public schools and for most variables, it is NCES’s goal that useful estimates may be 
made for individual States as well as for different degrees of urbanicity across States. In the case 
of private schools, NCES produces U.S. national-level estimates and does not produce State-by- 
State estimates. 

SASS was conducted with a periodicity (intersurvey interval) of three years for the first three data 
collection cycles of this ongoing repeated survey. The first study was conducted pursuant to 
Subtask 20B.3 under Task Order No. 20 (SASS Evaluation) of the U.S. Department of Education 
technical assistance contract with Synectics for Management Decisions, Inc. (Contract No. 
RN910600.01) in support of the National Center for Educaficm Statistics (NCES). The starting 
point specified by the statement of work for that subtask was a paper that appeared in the Journal 
of Economic Dynamics and Control (Smith and Barzily, 1982). That ps^r was based on W. 
Smith’s dissertation research (Smith, 1980) regarding the optimal choice of sample size and 
intersurvey time intervals for repeated socioeconomic surveys. The first study considered the 
relevance and potential applicability of previous work by W. Smith and others to the problem of 
optimizing the periodicity of SASS. It then developed and presented an alternative procedure for 
making an optimal choice of pmodicity based on an approach suggested by D. Ghosh which 
utilizes “probable error” models. The second study extends the work carried out in the first study, 
both in its theoretical and applied aspects. In particular, the second study has examined and 
refined five probable-error models and has considered periodicities of 4, 5, or 6 years. In addition, 
it has considered a set of specified scenarios for different proportions of variable cost to fixed 
cost — specifically, 30:70, 40:60, 50:50, 60:40, and 70:30. These proportions determine the 
amount available for the variable cost which, in turn, determines the size of sample that can be 
supported for each survey. 
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For the purposes of both studies the “optimal choice” of periodicity for SASS is assumed to be 
made in a conceptual framework that provides a means of taking into account the fixed and 
variable costs of operating a system of repeated surveys and the imputed costs resulting from the 
use of out-of-date estimates by public and private data users. The framework assumes that data 
collected annually (and processed with no delay) would represent the benchmark against which less 
frequent data collections (and lengthy processing times) would be compared in terms of data 
quality and cost. With the passage of time (due both to the choice of intersurvey interval and to 
data processing delays), estimates of variables of interest are assumed to have an increasing degree 
of uncertainty (error). The framework assumes that this growth in the estimation error can be 
quantified and that a resulting “cost” to the data user can be characterized. The goal, within this 
cost-and-error framework, is to select an intersurvey interval and a survey sample size that are 
jointly optimal. 

The current SASS series, with the Census Bureau as the collection agent for NCES, has now been 
conducted for school years 1987-88, 1990-91 and 1993-94. Other data sources, primarily the 
Common Core of Data (CCD) annual data collection, provide basic information for public schools 
on the number of schools (by type and grade levels), the number of teachers, and enrollments. 

SASS is a supplemental survey system designed “to measure the critical aspects of teacher supply 
and demand, the composition of the administrator and teaching work force, and the status of 
teaching and schooling generally.” Although the first three cycles of the series have been collected 
at three-year intervals, the frequency of data collection is not mandated by statute or departmental 
regulation. Tradeoffs between different periodicities (such as 4, 5, or 6 years) and corresponding 
sample sizes will now be subject to technical, policy, and budgetary review in NCES. 

There is an extensive literature on the analysis of repeated surveys which has developed over the 
past twenty years, primarily through the work of British and American statisticians. Section 3 of 
the present report includes a brief review of currently available time series methods for the analysis 
of repeated surveys and pertinent citations are included in the Bibliogrs^hy section. Some 
connections between and among classical exponential smoothing, Box-Jenkins time series 
modeling, Kalman filtering, state-space models, and signal extraction are outlined. The discussion 
recognizes the limitations of conventional modeling techniques and points out the possible need to 
accommodate level-dqjendent variances and perh^s other state dependencies in the modeling of 
schools and staffing survey data. 

Although both the first and second study depend in part on the prior work in Smith (1980), Smith 
and Barzily (1982), and Smith and Zalkind (1978), the Smith-Zalkind-Barzily optimization tools 
are not a fundamental basis for either of the two studies. The S-Z-B modeling ^jproach is not 
directly ^plicable to short duration time series with possibly time varying variance-covariance 
matrices. Nonetheless, the S-Z-B approach has provided a useful conceptual framework for the 
two studies and is incorporated in principle in one of the probable-error models (Model 4M) 
utilized in the present study. 

One of the present authors (D. Ghosh) is credited with formulating a family of “probable-error” 
models for use as a decision support tool in identifying optimal or near-optimal periodicities. The 
basic models and their extensions are summarized in section 2. The models are enable of 
recognizing the fixed and variable costs of repeated surveys and of minimizin g average error 
(“projected absolute error”) as a function of intersurvey time intervals (periodicity) and sample 
sizes. The analyses start with the assumption that total (multi-year) resoirrces are fixed and that the 
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p’oblem is to determine the best periodicity for the survey series in the light of average projected 
absolute error incurred over a multi-year cycle of repeated surveys. 

In the first study a 12-year cycle with intersurvey intervals of 2 years, 3 years, and 4 years was 
analyzed. Model 1 and two other models were then t^plied to each case. Tlie model testing 
involved the selection of twelve different items (“policy variables”) that were part of the SASS data 
for the 1987-88 and 1990-91 cycles of data collection. The 1993-94 data were not available at the 
time the first study was conducted, but have been utilized in the conduct of the second study. 

Tables were presented in the 1995 technical report which displayed the “total average error 
incurred in a 12-year cycle” for each of the selected variables under Models 1, 2, and 3 with 
periodicities of 2, 3, and 4 years. The fourth model, which was a variation of the S-Z-B t^proach, 
provided meaningful results for only three of the eleven variables. The assumptions for survey 
costs for the purposes of the previous study were described in section 5 of the 1995 report. Under 
the assumed total resource constraint, the resulting sample sizes were 8,667 for a 2-year interval, 
13,000 for a 3-year interval, and 17,333 for a 4-year interval. Under the cost assumptions, a 2- 
year periodicity was determined to be optimal under Models 1, 2, and 3. As noted in the body of 
the 1995 report, this optimality determination was based on minimizing total average error, subject 
to the stated cost constraints, for national-level data. It did not address the impact on State-level 
estimates. It is therefore important to assess any impact on State-level estimates since NCES 
intends to maintain its ability to publish directly estimated State-level figures for most SASS 
variables. The possibility of utilizing indirect estimates, such as estimates from time series models, 
is currently only of theoretical interest. Tliere are no present plans for implementing such 
estimates. 

The present technical report (“1996 report”) includes empirical results for public schools for three 
sets of State-level data. A preliminary analysis is provided for assessing the impact on State-level 
estimates of, for example, a decrease in overall U.S. sample size from approximately 9,000 to 
7,^)0 in moving from a projected five-year periodicity to a four-year periodicity. These particular 
hypothetical, but plausible numbers, arise from an analysis at the mid-point of a set of specified 
scenarios for different ratios of variable cost to fixed cost; that is, if the ratio of variable to fixed 
costs were 50:50 and a national sample of approximately 9,000 public schools could be supported 
under a five-year periodicity, then a four-year periodicity could only afford to sample 
approximately 7,200 public schools. 

For each of four probable-error models we have assumed that the total resources are fixed and 
periodicities of four, five, and six years were considered. As noted in the work statement for Task 
46, the “alternatives assume that the fixed cost bears a constant relationship to the variable cost.” 
As required by the task statement, we have considered the following different proportions of 
variable cost to fixed cost: 30:70, 40:60, 50:50, 60:40, and 70:30. Variable cost determines the 
total sample size for each round of SASS for each of the periodicities specified. Based on cost 
information from NCES, we have assumed an approximate total sample size of 9,000 for public 
schools and 2,500 for private schools for the five year periodicity when the variable cost is at the 
mid-point of the range, namely, at 50 per cent of total cost Tlus basic assumption fixes the sample 
sizes for all other cases because of the simple proportionality over all of the scenarios. 

The family of models developed assumes that the true value of a variable remains constant for a 
year after the survey date. The variable undergoes a change at the end of each year. The 
magnitude of this change is denoted by IDI . The sampling error component is the sampling 
absolute error , sae(n) , which is a function of the sample size n . 
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In Mcxlel 1 , we ignore the direction of the change in the value and just add the probable error to the 
successive changes in the true value. In Model 2A, we take into account all the possibilities and 
calculate the expected values over all the possibilities. For each successive year we calculate the 
expected value over all the projected possible values. In Model 3A, we assume the year-to-year 
process change to be a normal random variable with a nonzero mean. 

The projected absolute error is then calculated from a variable that combines the process change 
and the sampling error. In Model 4M, we explicitly assume that the process change which occurs 
each year is a Random Walk process in discrete time. Numerical results for public schools under 
Models 1 , 2A, 3 A, and 4M are set forth in Appendix A (for large-sample scenarios with 
periodicities of 4, 5, or 6 years) and in Appendix B (for an alternating large-and-small-sample 
scenario with a large-sample data collection every 6 years and a small-sample data collection 3 
years later) for public schools under Models 3A and 4M. 

1.2 Direct and Imputed Costs in the Choice of Survey Periodicity 

Any formalization of the problem of seeking an “optimal” choice of survey interval and survey size 
must account for the fixed and variable costs of operating a system of repeated surveys, such as 
SASS, as well as imputed costs due to increasing errors in the estimates as sample size is reduced 
and out-of-date estimates are used. In a recent book on survey errors and survey costs. Groves 
(1989) provides an up-to-date review of the kinds of considerations which should go into creating 
cost-and-error models for surveys, with particular emphasis on household surveys. Currently there 
is no comparable work on cost-and-error modeling for establishment surveys or surveys of 
institutions such as schools. 

In the case of SASS, there is an ongoing, more-or-less fixed, annual cost of maintaining the core 
elements of the SASS system whether or not a survey is conducted in a particular year. These 
ongoing costs are incurred both at the National Center for Education Statistics (NCES) and at the 
Bureau of the Census. Some costs might be regarded as either fixed or variable. Among these are 
the costs of updating list and area frames, with special emphasis on updates immediately preceding 
each cycle of data collection. In this report we lump such costs with the fixed annual costs of 
providing detailed institutional memory for all aspects of SASS (at NCES and at Census), making 
evolutionary design changes in coverage and content to be incorporated in the successive cycles of 
data collection, and conducting ongoing methodological research in support of SASS processing 
and estimation procedures. 

We assume that the variable costs can be summarized sufficiently well in a linear cost function 
with a constant unit cost coefficient so that the total variable cost for each cycle is a simple 
multiple of the number of schools in a data collection cycle. The reality is a bit more complicated 
than that, but within a real-world practical range of designated sample sizes for public and for 
private schools, it is not unreasonable to treat the survey unit cost per school as a fixed number of 
dollars, with or without adjustment for inflation. It would also be possible to set up the problem 
over, say, a 12-year or longer time horizon as a present value problem in line with the guidance the 
Office of Management and Budget prescribes for capital investment projects with a life-cycle cost. 

In addition to the directly measurable dollar outlays associated with maintaining and operating the 
SASS system, it is possible to include imputed dollar costs to represent the loss or penalty which is 
incurred by public and private users as a result of using outdated survey data. Smith and Zalkind 
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(1978), Smith (1980), and Smith and Barzily (1982) incorporated such an ^proach formulating an 
imputed loss associated with the use of imprecise estimates from an observed economic process 
where the objective was the allocation of public funds on the basis of such estimates. One 
development from this earlier work was the formulation of an “equivalent sample size” form of the 
Kalman filter. This is reviewed briefly in section 3 and section 5 of this report and involves a 
framework in which knowledge of the state of a socioeconomic process is characterized as the level 
of a stock of information (an equivalent sample size on hand). The original approach required a 
policymaker to select a scale factor or equivalence to characterize the “cost of not knowing” in 
dollar terms so that the imputed cost or loss can be combined in the same formulas with the dollar 
outlays. 

13 Notes on the First Three Rounds of SASS Data Collection 

The current SASS series, with the Bureau of the Census as the collection agent for NCES, was 
inaugurated for school year 1987-88 and repeated in 1990-91 and again in 1993-94. Although the 
first three cycles of SASS were conducted at three-year intervals, SASS has not had an officially 
established frequency of data collection. As a result of budgetary restrictions, the next round of 
SASS is currently scheduled to take place in 1998-99; that is with a periodicity of five years. 

The Schools and Staffing Survey is an overlapping survey (see Kalton and Citro, 1993, for a 
classification scheme for siuweys across time). For public schools the SASS design for the 1990- 
91 cycle provided approximately 30 per cent overlap for schools within each State with those 
selected in the 1987-88 cycle while using an updated list frame for the second cycle. For private 
schools, there was also a target degree of overlap nationally by type of private school or national 
affiliation. 

Kaufman and Huang (1993) describe the key design and implementation aspects of the 1990-91 
SASS in relation to the 1987-88 SASS. The survey is designed “to measure the critical aspects of 
teacher supply and demand, the composition of the administrator and teaching work force, and the 
status of teaching and schooling generally.” The sample sizes for public and private schools and 
administrators, local education agencies, and teachers were ^proximately the same for the two 
cycles of SASS. The core SASS components are the ‘Teacher Demand and Shortage Survey”, the 
“School Survey”, the “School Adnunistrator Survey”, and the ‘Teacher Survey”. 

There were changes in SASS design from 1987-88 to 1990-91. The public school frame for 1987- 
88 was the QED (Quality Education Data) file and for 1990-91 was the CCD (NCES’s Common 
Core of Data) school file. For private schools, the 1987-88 SASS used a dual-frame approach 
with a list frame based on the 1986 QED file and an area frame with the same primary sampling 
units used in the 1983 Private School Survey. For the 1990-91 SASS private school component, 
the 1989 Private School Survey (PSS) provided the sampling frame. 

Although the total public school sample sizes were the same for 1987-88 and 1990-91 (9330 
schools), the sample allocation methodology for 1990-91 differed in several respects from the 
methodology used for 1987-88. These changes were designed to improve the analytical usefiilness 
of the resulting survey data sets for making estimates at the State level without doing too much 
harm to national estimates. 

For public schools there was a controlled overlap of 30 per cent between 1987-88 and 1990-91. 
The QED definition was used to define the school for matching purposes at sample allocation time. 



Although overlap flags are included on the 1990-91 restricted-use SASS data sets, neither the 
APIN number (QED identifier) nor any equivalent matching identifier appear on both the 1987-88 
and 1990-91 restricted-use data sets. We could not ascertain an explanation for this discrepancy. 
For some desired purposes of the present study it was thus not possible to make an exact match of 
the overlapping public schools in any routine way. The change to the CCD definition for the 
public school portion of the 1990-91 SASS while using the QED definition for overlap matching 
purposes did result in some problems in maintaining the target sample allocation goals. The 
problem arises because some QED sites are the locus of more than one CCD school. 

In addition to the sampling design changes there were some questionnaire content and definitional 
changes. The variables in the two cycles that attempt to characterize the “type of community” 
caimot be directly reconciled. In 1987-88 there was a self-report by administrators of community 
type and in 1990-91 a “locale code” based on Census “urbanicity” information on population 
density, SMS A codes, and urban/rural area descriptors. 

The total sample size and the State-by-State sample allocations for the 1993-94 SASS differed 
firom the 1990-91 round in only minor ways. These differences did not present any problems for 
the conduct of the current study. Abramson et al. (1996) describe the changes in SASS design 
firom 1991 to 1994, including questionnaire and conceptual changes. 
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2. Probable-Error Methods as Decision Support for Choice of Periodicity 



In our previous and current work, we have assumed that data users will keep on using the 
Hata obtained from the most recent past survey until a new survey is undertaken and the 
newly collected data are processed and released to data users. Thus, if the intersurvey 
period is long, “deterioration” of the data could affect the quality of decisions made by 
users. On the other hand, if the survey is undertaken very frequently, the costs of 
conducting the survey and analyzing the data and the indirect costs of the response burden 
may be judged to have costs that exceed the benefits achieved in using current data. In the 
context of repeated surveys, it is useful to distinguish both opportunities and problems 
presented by different designs. 

Typical analyses of cost-benefit tradeoffs tend to focus on the best use of a fixed resource 
am ount over a time period that would include two or more survey data collections. The 
present budgetary restrictions for the 1990s are such that the “fixed” resource amount may 
be arbitrarily depressed and may overconstrain any realistic formulation of the 
optimization problem. In fact, the “truly optimal” formulation may be precluded by 
external constraints. 



2.1 Cost Models with Fixed and Variable Costs 

The usual cost model for a sample survey assumes a start-up cost Cq and a per unit 
(ultimate sample unit) cost Ci- Thus, the total cost is represented as C = Co + nCi . 
However, this start-up cost may be dependent on the periodicity. We represent it as Co 
(where k is the periodicity) which may be regarded as increasing with increasing 
periodicity; i.e., the start-up cost is more if the periodicity is three years compared to the 
start-up cost if the periodicity is two years and so on. On the other hand, the start-up cost 
may be considered to be constant; i.e., it does not depend on the periodicity. 

In the family of statistical models that we develop below, we assume that the total 
resources are fixed. The different possible periodicities spend these total resources in 
different ways. This assumption then determines the possible sample sizes every time the 
survey is undertaken corresponding to different periodicities. Thus, if we are comparing 
two possible periodicities, say two years as against three years, we consider a six-year 
cycle (the least common multiple of the two periodicity numbers). In the six-year cycle, a 
survey with periodicity two years will be conducted three times while a survey with 
periodicity three years will be conducted only twice. If C/ and Ci (where Ci is 
assumed to be independent of the periodicity of the survey) are known (whether the start- 
up cost is constant or increasing), we can calculate the possible sample sizes for these two 
alternatives where the total measure C is also known. A modified approach would be to 
use similar models but to attempt to take explicit account of the fact that total resources 
may be arbitrarily reduced by external constraints and formulate the decision problem 
within that framework. 
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2.2 Update and Extension of a Family of Probable-Error Models 

We assume that the true value of a variable remains constant for a year after the survey date. This 
is an appropriate assumption for the SASS survey system since nearly all of the observed variables 
under the various SASS questionnaires have an annual accounting period and the SASS data user 
is interested in changes in variables which are specified to change as of some conventional time 
point. For example, the official figure for enrollment and number of teachers in a public school is 
the fall (October) enrollment and the corresponding number of teachers or the full-time equivalent 
(h i t) number of teachers counted at the same point in time. The student enrollment and the 
teacher count may fluctuate during the academic year, but SASS and the Common Core of Data 
(CCD) are, in effect, taking sn£q)shots at the same time over a sequence of years. The error 
incurred in using a survey estimate is exactly equal to the difference between the survey estimate 
and the true value. Within the twelve-month interval from the survey date any user incurs an error 
which equals the difference between the true value and the survey estimate. The estimated standard 
error of the survey estimate provides an indication of this difference. The usual definition of the 
standard error for a statistic based on a sample from the population is that it is the “positive square 
root of the variance of the sampling distribution of a statistic” (Kendall and Buckland, 1971). The 
survey estimate is assumed to be approximately normally distributed around the true value with a 
standard deviation which is the standard error of the estimate. 

If one were interested in estimating from SASS data for a survey year the mean of some 
characteristic for a specified group of schools, such as the average “number of K-12 teachers that 
are new to the school this year” for all regular public elementary schools in the State of California, 
then the estimate would be constructed by jqjplying the school weight for each school to the 
reported number of new teachers for that school, summing the products and dividing by the sum of 
the school weights. For some of the SASS-based public school statistics published by NCES, such 
as those in the publication series Schools and Staffing in the United States: A Statistical Profile 
for each round of SASS, the NCES publications include tables of State-by-State estimates of the 
statistics and, for a selected subset of the State-by-State statistics, they also include tables of the 
estimated standard errors for those statistics. For example, the Statistical Profile for 1990-91 
includes, for public schools, both estimates of the statistics and estimated standard errors for these 
statistics on a State-by-State basis for (1) Number of public schools and students and average 
number of students per full-time-equivalent (FTE) teacher, (2) Percentage distribution of public 
school teachers by sex and race-ethnicity, percent mincffity teachers, and average teacher age, and 
(3) Average base salary for full-time public school teachers and average public school principal 
salary. As stated in the technical notes to that publication, “Standard errors were estimated using a 
balanced repeated replications procedure that incorporates the design features of this complex 
survey.” 

The difference between the true value and the survey estimate is the deviation from the mean m in 
the normal distribution of the survey estimate x considered as random variables. The average 
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of these deviations is called the mean (absolute) deviation. It is calculated as follows for any 
normal distribution: 




The projected absolute error incurred by any user during the first year after the survey is just the 



assuming simple random sampling. At the end of each year we assume that the true value 
undergoes a change. The magnitude of this change at the end of each year is denoted IDI . The 
sampling error component is the sampling absolute error, sae(n) . So the expected value of the 
total error incurred by all the users is dependent on sae(n) and on !D1 . The magnitude of the 
change at the end of the second year is also IDI , and so on. 

During the course of our work on periodicity conducted since 1993, the research team has explored 
several probable-error models (designated as Models 1, 2, 3, 4, 2A, 3 A, and 4M). In addition, a 
time series model (designated as Model 5) was treated briefly in Ghosh et al. (1994). Four of the 
probable-error models were selected for extended analysis through the application of three rounds 
of SASS data. These are described below. 

In Model 1, which was described in Ghosh et al. (1994), we ignored the direction of the change in 
the true value and just added the sampling absolute error sae(n) to the successive changes D in 
the true value. Model 1 is “ultra conservative” or “pessimistic” in that successive changes are 
assumed to result in the largest possible cumulative year-to-year drift away ftom the last observed 
value. 

In Model 2 A, which is a variant of Model 2 described in Ghosh et al. (1994), we took into 
account all the possibilities and calculated the expected values over all the possibilities. The 
change may be positive or negative; that is, the new value may be larger or smaller than the 
previous year’s value. We estimated the magnitude of change from the past data and used the 
estimate to project all possible new values. For each successive year we calculate the expected 
value over all the projected possible values. 

In Model 3A, which is a variant of Model 3 described in Ghosh et al. (1994), we assume the year- 
to-year process disturbance (process error) to be a normal variable with a nonzero mean. We 
assume that this mean is approximated by calculating the average change from the past data. 

Since the process error and the sampling error are both assumed to be normally distributed, they 
can be readily combined. The projected absolute error, or p.a.e . , is then calculated from this new 
combined normal variable. 

In Model 4M, which is related to both Model 4 and Model 5, described in Ghosh et al. (1994), we 
explicitly assume that the process change which occurs each year (for example, every October 1^‘) 
occurs in accordance with a Random Walk process in discrete time. In the original models we 
introduced the concept of a loss parameter that converted the sampling error together with the 
unobserved process shift in non-survey years to a loss, expressed in monetary units. The 
combination of average cost and average error over a period of years was minimized to detemine 
the optimum periodicity. This was a variation on the S-Z-B approach and was also a variation on 
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an analysis suggested by S. Kaufman. Mcxlel 4M does away with the need for a separate loss 
parameter, thus avoiding the introduction of a subjective judgment on the part of the survey 
administrator. In Model 4M the process equation is 



X, = x,.i + w, 

where w, has mean zero. We then calculate the average error for different possible periodicities of 
the repeated survey. The optimal intersurvey interval can be determined if the process variance 
and the sampling variance are known. In a Random Walk model, the current level x, of the 
process is the best current forecast for any future year. One assumes that any known trend 
component has already been subtracted out. In general, data users will tend to use the last 
available survey value as long as no new survey has been conducted. TTiis “common sense” 
practice is consistent with the assumption of an underlying Random Walk process. 

The following tables set forth the year-by-year evolution of the projected absolute errors for four 
models (Models 1, 2A, 3A, and 4M). For Models 1, 2A, and 3 A the evolution is based on IZ)I , the 
magnitude of the annual change in the true value, and the sampling absolute error, sae(n). For 
Model 4M, the evolution is based on , assumed to be equal to the variance of the process 
disturbance w, , and the sampling absolute error, sae(n). 

Under each of the present models, in Year 1 , which is the year in which the survey is conducted, 
the projected absolute error is just the sampling absolute error, sae( n). In Year 2 the process has 
exhibited a change of magnitude \D\ . Under Model 1 the process change is simply added to the 
sampling absolute error sae(n) from Year 1 to obtain the Year 2 projected absolute error. Under 
Model 2A, provided IZ)I > sae(n), the projected absolute error for Year 1 is still sae(n) but for 
Year 2 it is now just \D\ since, on the average, sae(n) vanishes. Under Model 3A, where the 
process change is assumed to be normally distributed, the Year 2 projected absolute error is just 
the sum of 0.8 \D\ and the sampling absolute error from Year 1 sae(n). Under Model 4M the 
Year 2 projected absolute error is the root mean square quantity [(0.8 D)^ + sae(nfY ^ , which is 
just the square root of the sum of the squares of the corresponding terms for Model 3 A. 
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Projected Absolute Errors for Selected Models — Models 1 and 2A 



Year 


Model 1 


Model 2A [for D> sae(n) ] 




1 


sae{n) 


sae{n) 


2 


\DWsae{n) 


IDI 


3 


2 \D \+sae(n) 


ID 1+0^ 5ae(n) 


4 


3 ID \+sae(n) 


I^IDI 


5 


4 ID \+sae(n) 


L5 ID hQ315sae{n) 


6 


5 ID \+sae(n) 


1.875 IDI 


Avg 

p.a.e. 


\D\+sae(n) 


— ^ — \DWsae{n) 






o ■' 
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Projected Absolute Errors for Selected Models — Models 3A and 4M 



Year Model 3A Model 4M 

1 

sae{n) sae{n) 


2 

O.S \Dh-sae(n) 0.8^ 


D‘ + 

_ 


5ae(n)7 

0.8 








O-Syfl ID U-sae(n) 0.8^ 


|2D' + 


sae(n) 

0.8 


-|2 






4 

0.8.^/J' \Dh-sae(n) 0.8^ 


3D' + 


sae(n) 

0.8 


]2 


5 

0.8-y/4lD h-sae(n) 0.8^ 


4D' + 


sae(n) 

0.8 


n 2 


6 

O-Sy/S \ DU- sae(n) 0.8^ 


5D' + 


sae(n) 

0.8 


2 


p.a.e. IDh-iac(n) My (i-nn^ + 

6 6t!f ' 


5ae(n)"|^ 

0.8 
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2.3 Model Results Based on Three Rounds of SASS Data 



In consultation with NCES staff, we had previously identified several variables deemed important 
to SASS data users. We then applied the models developed above using three rounds of SASS 
at the national level (U.S.) for public and private schools and at the State level for public 
schools in three selected States (California, Iowa, and New York). The following twelve variables 
were selected from the School, Administrator, and Teacher questionnaires: 



Item 1. 
Item 4. 
Item 6. 
Item 7A. 
Item 7B. 
Item 8A. 

Item 8B. 

Item 9. 
Item lOA. 

Item lOB. 

Item 11 A. 

Item IIB. 



Number of students served by chapter 1 services (Schools — public). 

Number of K-12 teachers that are new to the school this year (Schools — public). 
Proportion (%) of all schools with minority principals (Adminr — public & private). 
Number of students per FTE teacher (Schools — public). 

Number of students per FTE teacher (Schools — private). 

Proportion (%) of schools in which various programs and services were available to 
students (Schools — ^public). 

Proportion (%) of schools in which various programs and services were available to 
students (Schools — private). 

Proportion (%) of principals having master’s degree (Administrator — public). 

Proportion (%) of full time teachers who received various types of compensation, by 
sector (Teacher — public). 

Proportion (%) of full time teachers who received various types of compensation, by 
sector (Teacher — ^private). 

Proportion (%) of full time teachers who were newly hired and who were first time 
teachers (Teacher — ^public). 

Proportion (%) of full time teachers who were newly hired and who were first time 
teachers (Teacher — ^private). 



Private school items 7B, 8B, lOB, and IIB were omitted from the State-level computer runs since 
State-level estimates are not published by NCES for private schools. Item 6, which is based on 
pooled data for public and private schools combined, was retained in all runs. 

We obtained approximate estimates for the fixed cost and the variable cost elements of SASS. We 
applied the four models for each variable listed above, and computed the projected absolute errors 
for large-sample scenarios with periodicities of four, five, and six years and a range of variable- 
tofixed cost ratios. An alternating large-and-small-sample design was also calculated and 
analyzed. For selected models, the detailed results, estimates, and projected absolute errors are 
presented for public schools and private schools in the appendix tables of Part II of this report 
The detailed results include large-sample scenarios with periodicities of four, five, or six years and 
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variable-to-fixed cost ratios of 30:70, 40:60, 50:50, 60:40, and 70:30 and an alternating large- 
and-small-sample design with a large-sample data collection every six years and a small-sample 
data collection three years after each large-sample data collection. Numerical results for public 
schools are presented for national (U.S.) data and for the States of California, Iowa, and New 
York. Results for private schools are presented at the national level. 

2.4 Comments on Results for Large-Sample Scenarios (Models 3A and 4M) 

In the course of our theoretical and empirical analyses of the family of probable-error models, we 
found that the “discriminatory power” of the probable-error models to reveal meaningful 
differences between neighboring choices of periodicity was greatest for the combination of large 
sample size, which entails small sae(n ) , together with relatively large year-to-year process change 
D . Very large sample sizes are found only in the SASS public-school datasets for the U.S. as a 
whole. In this report, unless otherwise noted, sample size denotes the size of the actual obtained 
sample (net of nonresponding units). At the State level, public school sample sizes are in the low 
hundreds. The following tables and charts provide a consolidated siunmary of model results for 
Models 3A and 4M for the U.S. and for the States of California, Iowa, and New York. In these 
tables and charts the range of periodicities has been extended for expository purposes to a range of 
2 to 6 years, while the appendix tables are restricted to a range from 4 to 6 years, which is the 
range of practical operational interest to NCES under current budgetary restrictions. 

For the U.S. as a whole, the projected base case (5-year periodicity and variable-to-fixed cost ratio 
of 0.5) sample sizes are 9,000 public schools and 48,000 public-school teachers. With these large 
sample sizes, the averages of the relative projected absolute errors (where the p.a.e.divided by the 
estimate of its variable is the rel p.a.e.) for the eight public-school variables under both Model 3 A 
and Model 4M are smallest for short periodicities and largest for longer periodicities even though 
sample sizes are markedly smaller for the shorter periodicities. The effect of different cost ratio 
assumptions on the rel p.a.e. was not large. For large-sample scenarios the average rel p.a.e. is 10 
to 20 per cent better for a cost ratio of 70:30 than for 30:70. 

For California the base case sample sizes are 350 public schools and 2, 100 public-school teachers. 
For New York the corresponding sample sizes are 270 schools and 1,460 teachers. For Iowa the 
sample sizes are 160 public schools and 900 teachers. For both California and Iowa, under both 
Model 3A and Model 4M, the averages of the relative projected absolute errors are larger for short 
periodicities and smaller for longer periodicities. For New York, the mean values of the relp.a.e. 
are essentially flat under Model 3A over the periodicity range from 2 to 6 years but under Model 
4M the values decline initially, with a minimum at a periodicity of 4 years, and then rise slightly. 

Under the probable-error models the data users who are primarily interested in carrying out 
analyses for individual States will generally incur smaller errors if they are provided with datasets 
from longer periodicities and hence larger sample sizes. Data users who are primarily interested in 
carrying out analyses for the U.S. as a whole will incur smaller errors if they are provided with 
datasets from shorter periodicities and correspondingly smaller sample sizes. Again, it is important 
to note that the mean rel p.a.e. values do not exhibit large differences over the range of 
periodicities. Nonetheless, State-level analyses are generally favored by longer periodicities and 
their correspondingly larger sample sizes. 
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SASS Data and rel p.a.e. for Selected Policy Variables 
under a Fixed Total Cost and Cost Ratio p=0.5 
U.S. - Public Schools 



Item 


SASS estimates 
Source 


Pooled 

Mean 


Estim 
Std Dev 


Avg D 


Avg D% 


RefN 


#1 


School (Pub) 


104.591 


150.153 


3.855 


3.7% 


9,000 


#4 


School (Pub) 


3.214 


4.000 


0.175 


5.4% 


9,000 


#6 


Administrator (Pub & Pri) 


0.095 


0.292 


0.003 


2.9% 


11,500 


#7A 


School (Pub) 


16.979 


5.280 


0.234 


1.4% 


9,000 


#8A 


School (Pub) 


0.993 


0.084 


0.001 


0.1% 


9,000 


#9 


Achninistrator (Pub) 


0.972 


0.166 


0.004 


0.4% 


9,000 


#10A 


Teacher (Pub) 


0.386 


0.486 


0.010 


2.5% 


48,000 


#11A 


Teacher (Pub) 


0.034 


0.181 


0.001 


3.5% 


48,000 





Model 3A 




Rel p.a.e. Mod 3A p=0.5 
“ Periodicity — 




Item 


Source 


2 vrs 


3 vrs 


4 vrs 


5 yrs 


6 vrs 


#1 


School (Pub) 


2 

0.0339 


3 

0.0394 


4 

0.0441 


5 

0.0484 


6 

0.0523 


#4 


School (Pub) 


0.0384 


0.0486 


0.0569 


0.0640 


0.0704 


#6 


Achninistrator (Pub & Pri) 


0.0481 


0.0484 


0.0498 


0.0516 


0.0534 


#7A 


School (Pub) 


0.0096 


0.0122 


0.0143 


0.0161 


0.0178 


#8A 


School (Pub) 


0.0014 


0.0014 


0.0014 


0.0014 


0.0014 


#9 


Achninistrator (Pub) 


0.0041 


0.0047 


0.0053 


0.0058 


0.0063 


#10A 


Teacher (Pub) 


0.0173 


0.0221 


0.0260 


0.0293 


0.0323 


#11A 


Teacher (Pub) 


0.0448 


0.0476 


0.0508 


0.0538 


0.0568 


Avg 




0.015 


0.017 


0.019 


0.021 


0.022 





Model 4M 




Rel p.a.e. Mod 4M p=0.5 
- Periodicity -- 




Item 


Source 


2 yrs 


3 yrs 


4 yrs 


5 yrs 


6 yrs 


#1 


School (Pub) 


2 

0.0295 


3 

0.0312 


4 

0.0357 


5 

0.0400 


6 

0.0441 


#4 


School (Pub) 


0.0337 


0.0407 


0.0489 


0.0563 


0.0630 


#6 


Administrator (Pub & Pri) 


0.0444 


0.0373 


0.0375 


0.0388 


0.0406 


#7A 


School (Pub) 


0.0085 


0.0103 


0.0124 


0.0142 


0.0159 


#8A 


School (Pub) 


0.0013 


0.0011 


0.0010 


0.0010 


0.0011 


#9 


Administrator (Pub) 


0.0035 


0.0038 


0.0043 


0.0048 


0.0053 


#10A 


Teacher (Pub) 


0.0152 


0.0186 


0.0225 


0.0259 


0.0290 


#11A 


Teacher (Pub) 


0.0400 


0.0365 


0.0389 


0.0418 


0.0449 


Avg 




0.013 


0.014 


0.015 


0.017 


0.018 
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Average rel p.cue. for Selected Policy Variables, Fixed Total Cost and p=0.5 



Model 3A 





Model 4M 
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2.5 Comments on Results for An Alternating Large-and-Small-Sample Design (Models 

3A and 4M) 

The previous section has discussed the modeling results for large-sample scenarios with various 
intersurvey intervals. Detailed results for scenarios under Models 1, 2A, 3A, and 4M are set forth 
in Appendix A and Appendix C for public schools and private schools respectively. Detailed 
analysis of those results has led in turn to the formulation of an alternating large-and-small-sample 
design. The results for a specific hypothetical implementation of this design approach are set forth 
in Appendix B and Appendix D for public schools and private schools respectively. 

A specific “Interleaved 3-6-9 Design” was formulated as follows; First , assume the same fixed 
annualized resource budget that would otherwise support the large-sample scenarios with a 
periodicity of five years over a range of cost ratios (30:70, 40:60, 50:50, 60:40, 70:30). Then the 
assumed obtained sample sizes for the U.S. at the mid-point 50:50 cost ratio were 9,000 public 
schools and 48,000 teachers. These sizes were proportionally smaller or larger for smaller or 
larger cost ratios. Next, assign these sample sizes to a periodicity of six years instead of to a 
periodicity of five years. This results in a “cost dividend” of 20 per cent which is then invested in a 
one-fifth U.S. sample of 1,800 public schools and 9,600 teachers for a data collection which can 
be conducted at the halfway point between two full -sample data collections; namely, three years 
after the previous large data collection. Assume that there is no processing delay. For simplicity, 
also assume that the schools in the one-fifth sample are nonoverlapping with the schools in the full 
sample. Further assume that for the U.S. as whole only direct estimates will be of interest and, 
hence, the two independent samples (the full sample and the one-fifth sample) will be treated as 
independent cross-section surveys three years apart. 

Now consider the projected absolute errors that will be incurred by a data user over a six -year 
period. During the first, second, and third years after a full-sample data collection a data user who 
is interested in national data will continue to use that sample. Under either Model 3A or Model 4M 
the projected absolute errors will increase each year. In the fourth year after the large sample data 
collection a new dataset from the one-fifth size national sample will become available. The data 
user then disregards the data in the old large sample and begins to use the data from the new one- 
fifth s am ple, and continues to use it until data from the next full sample becomes available in the 
seventh year. For the U.S., the sample sizes in the one-fifth national sample are large enough that 
quite good estimates may be made. That is the user is not heavily penalized in shifting every three 
years between the full national sample and the one-fifth national sample. For the cost ratio p=0.5 
an inspection of the tables in Appendix B for each of the eight policy variables, and of the average 
rel p.a.e. values over six years for each of these variables as displayed in the U.S. sununary sheet, 
reveals in a comparison with the corresponding Appendix A tables that the average rel p.a.e. 
values for the interleaved “3-6-9” design with large sample periodicity of six years are less than or 
equal to the average rel p.a.e. values for the single large-sample scenario with periodicity of five 
years. Furthermore, the user of U.S.-level data will be receiving the benefits associated with the 
receipt of fresh data every three years. 

Referring now to the tables in Appendix B, we see that some of the sample sizes for public schools 
in the State portion of the one-fifth national sample (“3-6-9” Design) have shrunk to rather small 
sizes, ranging (for p=0.5) from 32 schools in Iowa to 70 schools in California. The teacher sample 
sizes have also shrunk, now ranging (for p=0.5) from 180 teachers in Iowa to 420 teachers in 
California. Although the user of State-level data might choose to ignore the State portion of the 
National sample, we have assumed for the purposes of the Appendix B tables that the user would 
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opt to make use of an ^propriate composite estimator based on the joint use of the two 
independent datasets from the State portion of the full sample and from the State portion of the 
one-fifth sample. Accordingly, the tables for each policy variable contain, for Model 3A and 
Model 4M, a State composite subtable of p.a.e. values in which the values for the fourth, fifth, 
and sixth years are root mean square composites of the p.a.e. values from the full sample and the 
one-fifth sample. 

For public school variables in Iowa, for example, the “3-6-9” composite average rel p.a.e. values 
are slightly higher than the corresponding Appendix A values for a full sample with periodicity of 
five years. For teacher policy variables the composite average rel p.a.e. values are about equal to 
the Appendix A values for a full sample with periodicity of five years. 

The user of the State-level datasets from the full sample and the one-fifth size small sample might 
alternatively opt for the explicit use of one of the time series modeling techniques described in 
section 3 below to construct minimum mean squared error estimates. Further analysis of such 
procedures is beyond the scope of the present report. 

The following table provides a national-level. Public Schools example of the modeling results (for 
Model 3A) under an alternating large-and-small sample design (the Interleaved 3-6-9 Design) with 
selected cost ratios of 30:70, 50:50, and 70:30. Further results and underlying detail may be found 
in Appendix B. 



Relative Projected Absolute Errors under Model 3A 
National (Public Schools) Interleaved 3-6-9 Design 
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0.972 


0.166 


0.004 
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0.005 


0.005 


0.004 


#10A 


Teadier (Pub) 
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48,000 


0.021 


0.019 
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0.034 


0.181 


0.001 


48,000 
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0.044 
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0.019 


0.016 


0.015 



The following table provides a national-level. Private Schools example of the modeling results (for 
Model 3A) under an alternating large-and-small sample design (the Interleaved 3-6-9 Design) with 
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selected cost ratios of 30:70, 50:50, and 70:30. A comparison with the summary table in Appendix 
C for a private school scenario with a periodicity of 5 years and a 50:50 cost ratio shows an 
average rel p.a.e. of 0.040 versus 0.039 in the table below; that is, they have essentially identical 
average rel p.a.e. values. Htus there is no penalty in terms of the average rel p.a.e. values as a 
result of using the alternating design versus the conventional periodic design and there is added 
value in that fresh data are available every three years. Further results and underlying detail may 
be found in Appendix D. 

Relative Projected Absolute Errors under Model 3A 

National (Private Schools) Interleaved 3-6*9 Design 
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Source 


Pooled 
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0.269 
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0.016 
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0.052 
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Teacher (Pub) 
















#11B 


Teacher (Pri) 


0.065 


0.246 


0.004 


8,000 


0.100 


0.084 


0.075 


Avg 












0.046 


0.039 


0.035 



2.6 Uses and Limitations of Existing Probable-Error Modeling Methods 

We have noted above many of the simplifying assumptions made in the construction and use of the 
family of probable-error models. In the first three rounds of SASS, conducted at three-year 
intervals, approximately 30 percent of the schools were in successive rounds. This partial overly 
was an explicit part of the SASS design. It does, however, induce a round-to-round COTrelation 
between successive observations on the overlap schools. Any effects of such correlations have not 
been dealt with in our existing probable-error models. In the case of the time series modeling 
approaches discussed in section 3 below, there are available techniques for dealing explicidy with 
correlated errors and other effects of overlap. 

In geno'al. Model 3A and Model 4M have given generally consistent results in profiling the 
approximate magnitude and direction (increasing or decreasing with periodicity choices) of the 
projected absolute errors for selected policy variables and combinations thereof 
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3. Time Series Methods in Relation to the Periodicity Problem 



3.1 Background on Time Series Methods for Repeated Surveys 

Since the early papers of Scott and Smith (1974) and Scott, Smith, and Jones (1977), there has 
been a renewed and growing interest in the application of time series methods to survey data. An 
excellent review article. Binder and Hidiroglou (1988), may be foimd in volume 6 of the Handbook 
of Statistics. This review and the papers by Bell (1984), Bell and Hillmer (1990), and Tam (1987) 
provide a balanced account of time series approaches, including state-space modeling and Kalman 
filter techniques, for use with data from repeated surveys. Although most statisticians are now 
aware of the time series methods of Box and Jenkins (1970), who provided an understandable, 
systematized approach to model identification, estimation, and forecasting, many survey 
statisticians are still imaware of the potential of the time series methods for improving estimation 
with survey data in the sense of minimizing mean squared error. The key principle in the time 
series approach is that there is information in the time series structure of an observed process 
which may be used to make better estimates by combining information from past data collections 
with the new information from a current data collection than would be made if the current data 
were to be used alone. 

Classical survey estimates are made under assumptions that the observed variables, whether of 
labor force, or school enrollment, or other socioeconomic phenomena, have values that are fixed 
but are observed with sampling error (and possibly nonsampling error). The time series approach 
regards the process variables as stochastically varying over time and the identification problem is 
to find a parsimonious time series model evolving in discrete time such as one-year intervals, that 
will capture the main features of the underlying process sufficiently well. For univariate processes 
it has often been found to be quite satisfactory to fit a model with a very simple structure, such as 
an autoregressive model of order 1 or 2. 

In the case of surveys of schools and similar institutions, the natural accounting period is the 
school year. Within-year changes are of secondary interest and seasonal effects do not arise. 

Linear trends are easily incorporated in Box-Jenkins type models and can be factored out by taking 
first differences of successive observations. Thus the trend component may be accounted for 
separately. One useful model that is related to classical exponential smoothing is the Box-Jenkins 
ARIMA(0,1,1). It is mildly nonstationary and can “wander” up and down; in one sense the current 
process value serves as a “local mean” for the process as time moves ahead one step and the 
process noise term kicks the process up or down a bit. In classical Box-Jenkins modeling it is 
assumed that the process is observed without observation error. 

Borrowing from the contributions of R. Kalman in the control engineering literature in the 1960s, 
the Scott-Smith time series approach utilizes a two-equation setup in which there is a process 
equation which represents the evolution of the imderlying (unobserved) process through time. The 
second equation, the observation equation, consists of the sum of the underlying process variable 
and an observation noise term. The noise term in some simple models may just represent the 
sampling error. In other cases it may have a structure of its own. The state of the process may be 
represented by a vector with two or more components, representing, for example, the levels of two 
or more process variables such as number of teachers and number of students at a school, or in an 
aggregate of schools within a State or other subnational grouping. 
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For example, if we assume that an m-component state vector x(t) evolves according to a vector 
random walk, we may write for the process equation 

x(t+l) = x(t) + w(t) , 

where w(t) is the process noise and is distributed as a multivariate normal, N(0, Q), with a zero 
mean vector and known process disturbance covariance matrix Q, where 
Q = E[ w(t) w’(t) ], t = 0,1,2,.... The classical Kalman approach is to assume that Q is known 
and is time invariant. 

The process variable x(t) is sometimes referred to as a “signal” which can only be observed in 
the presence of observation noise. The task is then to “extract” the signal from the surrounding 
noise. We then write an observation equation 

y(t) = x(t) + u(t) , 

where u(t) is the observation noise and may simply represent sampling error or might represent a 
more complicated form of measurement error. We may postulate a unit noise covariance matrix R 
for an observation on a sample of size one and a sample noise covariance matrix B(t) obtained by 
dividing the elements of R by a sample size N(t). For the simple random walk model, the optimal 
estimate between surveys is just the last previous estimate, say x^^(t 1 1) , given all information up 
through time t. That is, 

+ j 1 1) = x'^Ct 1 1) . 

When a new survey is taken (an observation is made), with an intersurvey interval of T years, the 
optimal estimate at that instant will be 

x'^Ct 1 1) = x'^Ct 1 1 - T) + K(t) [ y(t) - x'^Ct 1 1 - T) ] , 

where K(t) is known as the Kalman gain and is obtained from operations on the variance- 
covariance matrices for the process noise and the observation noise. 

Thus it may be seen the new estimate is a weighted combination of the new observation and the 
optimal estimate from the most recent previous survey cycle. There are certain details, such as the 
need to initialize the process state at time zero, but this simple example shows the essence of the 
signal extraction approach via the Kalman filter. 

The classical Kalman ^proach assumes that the variance-covariance matrices associated with the 
process equation and the observation equation are known and time invariant. In real world 
settings, these matrices will not be known and will have to be estimated from the data. 

Furthermore, they will not necessarily be time invariant. These complications have led to the 
formulation of extended Kalman filters. The extended filters place an estimation burden on the 
available data and may lead to inconclusive results. Also, it is somewhat awkward to try to 
accommodate nonlinear features such as the presence of level-dependent variances. For example in 
a set of elementary schools arranged by size within one State, the variance in enrollment or in 
number of teachers will typically depend on the size of the school and hence the number of 
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teachers. This is easier to capture using one of the model types known as state-dependent models 
and in particular with a class of models known as bilinear models (see Smith, 1994). 



3.2 Concept of Equivalent Sample Size in Kalman Filtering for Repeated Surveys 

In Chapter II of Smith (1980) the concept of equivalent sample size was introduced in the context 
of Kalman filtering for repeated surveys and adapted to a reformulation of the optimal filter 
theorem for a scalar (single- variable) model (which was designated as “Scalar Model 1”) of an 
evolving process observed at discrete points in time. A similar description of that development also 
appears in Smith, Ghosh, and Chang (1995). 

See section 5.1 below for a technical account of the development of an equivalent-sample size form 
of the Kalman filter and its application to random walk models of repeated surveys. In connection 
with that development. Smith and Barzily (1982) noted that “a survey administrator who was 
concerned that the tmderlying process parameters may take unexpected jumps or exhibit turning 
points, which are not modeled by the simple time-invariant random walk models, would 
presumably cpt for sampling more frequently than the optimal interval found by this method.” 



3.3 An Exploration of Modeling with Joint Use of CCD and SASS Data 

One of the consequences of the popularization of the Box-Jenkins methods is that many statistical 
practitioners may assume that one caimot use time series methods uidess one has a long time series 
of 40 to 60 observations. T.W. Anderson (see Anderson, 1978) showed that replicated cross- 
sectional observations with a sizable number of repeated measures at a few points in time can 
substitute quite well for single observations stretched out over time. While there are limitations to 
this approach, Azzalini (198 1) and others have recognized the value in this ^proach. In some 
cases it may be implemented rather simply with standard statistical software. 

Shumway (1988) t^plied the replicated autoregressive model of Anderson (1978) to a problem of 
forecasting grade-by-grade enrollments for a group of public elementary schools. He used four 
years of kindergarten-through-grade 6 data for a set of 12 schools and modeled the school 
enrollments as a vector autoregressive process of order 1. We provisionally adopted Shumway’ s 
general approach and investigated whether or not it could be extended to assess the possible 
dependence of components observed in the SASS periodic series, but urwabserved in the Common 
Core of Data (CCD) series, on the observed components in the aimual CCD series. Thus some 
variables were only present in the SASS series and were only observed every three years and some 
variables, such as grade-by-grade enrollment and teacher counts, were present in both series and 
moreover were observed every year in the CCD series. Shumway and Stoffer (1982) included 
some techniques for using the EM algorithm for estimation in both the complete-data case and in 
the case of irregularly observed data. 

For a test data set of eight California public elementary schools we constructed year-by-year 
process and observation state vectors with 16 components. These components included 
erffollments for kindergarten through grade 8, enrollment coxmts for ungraded students, full-time 
equivalent teacher counts, per cent minority enrollment, number of school lunch eligibles, number 
of school lunches served, chapter 1 services provided, and a component to be assigned. The first 
12 components were observed in the annual CCD series, the 13"' component (school lunch 
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eligibles) was observed in both CCD and SASS, and the last three components were observed, if at 
all, only in SASS and only at three year intervals. 

The theoretical formulation of a Markovian state-space model is described in section 5.2 of the 
Technical Supplement. In principle, the state-space approach carried over smoothly from the case 
of regularly (uniformly) spaced observations to the case of irregularly observed processes. 

Suitable recursive computational methods were available from the results in Shumway and Stoffer 
(1982). These included the steps needed for maximum likelihood estimation using the EM 
algorithm for the complete-data case as well as the extensions for the irregularly observed data 
problem. 

In practice, the available test data sets were not adequate for the planned analyses. In particular, 
for variables other than enrollment, it was found that variables, such as school lunch variables, that 
should have been observed every year in CCD were not in fact present in the database for every 
year and variables such as chapter 1 services were not uniformly reported in SASS. Where the 
analyses could be performed, for a small subset of schools, it was found that the dependencies of 
SASS variables on CCD variables, as shown by correlation matrices, were neither strong nor 
stable. No further work was pursued on these methods beyond these initial investigations. 



3.4 Uses and Limitations of Time Series Modeling Approaches 

In the proceedings p^)er (Ghosh et at, 1994) and technical report (Ghosh et ai, 1995) covering 
our previous study one of the present authors (D. Ghosh) set forth some new alternative models in 
the form of “probable-error” models which appeared to provide decision support for optimal choice 
of sample sizes and intersurvey interval, using data for a subset of variables at the geogrJ^)hic level 
of interest, without detailed model fitting of the underlying observed processes. Our current work 
with this family of models has been reviewed in section 2. 

Consideration of indirect (time series) estimation methods as an alternative to this direct 
approach was summarized Smith, Ghosh, and Chang (1995). That discussion also led to 
the investigation described described in section 3.3 concerning the possible ^plication of 
Shumway’s Markovian grade-by-grade analysis to joint estimation procedures utilizing 
CCD and SASS data together. As stated there, the results of that investigation were 
inconclusive. Furthermore, there is no current operational requirement by NCES for the 
development of indirect estimation methods. It is our understanding that NCES will 
continue to adhere to its current practice of using only direct estimates from survey data 
for production and publication of national and State-level results from SASS. 

As noted in section 2, the user of State-level SASS datasets, especially under the “3-6-9” 
interleaved design for alternating large-and-small samples, might wish to explore the 
feasibility of applying one or more of the time saies modeling techniques described above 
in order to obtain minimum mean squared error estimates for selected policy variables. 
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4. Conclusions and Recommendations 



4.1 Lessons from Probable Error Modeling and Time Series Approaches 

The probable-error models described in section 2 of this report are based on a number of 
simpbfying assumptions. Section 3 of the present report summarizes other available methods for 
modebng and analysis of repeated surveys, particularly time series methods stemming from the 
early work of Blight and Scott (1973) and Scott and Smith (1974) as well as the more recent 
“signal processing” approach of Bell (1984), Bell and Hillmer (1990), and others. Section 3 also 
reviewed the uses and limitations of one implementation of the Scott-Smith- Jones time series 
approach, namely the Smith-Zalkind-Barzily optimization tools first set forth in detail in Smith 
(1980). The results on optimal choice of periodicities for the Schools and Staffing Survey 
provided in the present report were obtained by qjplying Models 1, 2A, 3A, and4M to 1987-88, 
1990-91, and 1993-94 SASS data. 

Some users of national-level data would be better off with more frequent data availability even if 
the estimates were based on reduced sample sizes. One difficulty is that “publishable estimates” 
and “usable microdata” depend on which survey variables or construct variables are of interest to a 
particular data user. Furthermore, once the SASS databases provided to users are constructed 
from a “smaller but more frequent” SASS design, some State-level analyses could no longer be 
conducted using simple cross-sectional weights, but would have to take into account the more 
complex design elements. 

The main research effort carried out in the present study has been directed at refining and 
extending the probable-error models developed under the previous study and applying these models 
to data from three rounds of SASS (i.e., for school years 1987-88, 1990-91, and 1993-94). In the 
previous study, national data from two rounds of SASS were analyzed using four different 
probable-error models with alternative intersurvey intervals (i.e., periodicities) of 2, 3, or 4 years. 

In the current study, national and selected State-level data from three rounds of SASS have been 
analyzed (for the U.S. and for California, Iowa, and New York) using four different probable-error 
models with alternative intersurvey intervals of 4, 5, or 6 years and five specific scenarios for 
ratios of variable-to-fixed costs within an assumed budget constraint. 

Each analysis was replicated over data for 12 different policy variables that had been specified by 
NCES for use in the previous study and in the current study. There were four variables from the 
SASS public school questionnaire (Items 1,4, 7 A, and 8A), two variables from the private school 
questionnaire (Items 7B and 8B), two items each from the public (Items lOA and 1 1 A) and private 
(Items lOB and IIB) teacher questiormaires, respectively, one item from the public admini strator 
questioruiaire (Item 9), and one item analyzed on a combined basis for public and private 
administrators (Item 6). 

The base case was assumed to be an intersurvey interval of 5 years at a 50:50 ratio of variable-to- 
fixed cost. The dollar amoimt was assumed to be a five-year budget equal to the average of the 
previous two three-year budgets that were available when the intersurvey interval was 3 years. 

This has provided a base case sample size of approximately 9,000 public schools and 2,5(X) private 
schools for the school data collections at the assumed 5-year intervals and 50:50 cost ratio. For the 
associated teacher data collections the base case sample sizes are 48,000 public school teachers 
and 8,000 private school teachers. The base case sample size for principals/headmasters is then 
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1 1 ,500 for public and private schools combined. The sample sizes are perturbed in a linear fashion 
for 5-year interval cases over the variable-to-fixed cost ratios of 30:70, 40:80, 50:50, 60:40, and 
30:70. That is, at 30:70 the base case public school sample size becomes 5,400 and at 70:30 the 
public school sample size becomes 12,600. When the intersurvey interval is reduced from 5 to 4 
years, the public school sample sizes are reduced to 4,320 at 30:70, 7,200 at 50:50, and 10,080 at 
70:30. Similarly, when the intersurvey interval is increased from 5 to 6 years, the public school 
sample sizes are increased to 6,480 at 30:70, 10,800 at 50:50, and 15,120 at 70:30. 

The three rounds of SASS data collection provided estimates, at national and State level, of the 
average change in each policy variable over the two past three-year intervals. Pooled means and 
estimated population standard deviations were also constructed for each policy variable. Projected 
absolute errors and relative projected absolute errors (or rel p.a.e . ) were then computed, at the 
national level (U.S.) year by year, for the selected probable-error models (Models 3A and4M), for 
each of 12 policy variables, for each of 3 periodicities, and for each of 5 cost-ratio scenarios. For 
three selected States (CA, lA, and NY), the same runs were made with the exception that the 
private-school variables (Items 7B, 8B, lOB, and 1 IB) were excluded since NCES does not 
publish State-level estimates for private school variables. As noted in section 2, the effect of 
different cost ratio assumptions on the projected absolute errors was not large. 

Referring to the Appendix A summary sheet table entries for projected absolute errors under each 
of the Models 1, 2A, 3 A, and 4M and a 50:50 cost ratio for Public Schools data collections at the 
national level (U.S.) we see that the choice of a 4-year periodicity typically results in lower average 
projected absolute errors (and rel p.a.e.) than does a 5-year periodicity, and that a 5-year 
periodicity yields lower errors than does a 6-year periodicity. State-level projected absolute errors 
were essentially flat over the periodicity choices of 4, 5, or 6 years although in some cases the 
longer periodicities resulted in slightly lower rel p.a.e. values. 

Similarly, from Appendix C for Private Schools (U.S. national level), using the same four models, 
we see that the choice of a 4-year periodicity results in lower average projected absolute errors than 
does the choice of longer periodicities. 

For most sample sizes and most of the tested policy variables, the probable-error models do not 
sharply discriminate between adjacent choices of periodicity. This outcome parallels some results 
in Smith (1980) where it was noted that the minima for the imputed cost models used in that work 
were not sharply different for neighboring choices of periodicity. 

Again, it should be noted that Model 3 A does involve an assumption of normality for the process 
disturbance. For some variables of interest this assumption may appear to hold, at least 
approximately, while for others it may not. Model 4M has a number of attractive properties and is 
a truly stochastic model. The assumption of a Random Walk process for a discrete time series 
estimated from a sequence of repeated cross-section surveys has often proven useful. 

In general. State-level analyses are best supported by longer periodicities, while U.S.-level analyses 
are best supported by shorter periodicities. In summary, the probable-error models have provided 
a direct, approximate method for characterizing the problem of making a joint choice of intersurvey 
interval and sample size under a fixed annual cost constraint. 
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A Recommended Structure for Future SASS Data Collections 



The main recommendation from the present study is that the National Center for Education 
Statistics should consider adopting an Alternating Large-and-Small-Sample Design for SASS with 
an appropriate full-sample periodicity (such as six years) together with a mid-period fractional- 
sample data collection to be conducted half way between the full-sample data collections. The 
mid-period data collection would provide a periodic update at the national level and for the larger 
States. 

For Public Schools (national level), with an assumed 50:50 cost ratio under both Models 3 A and 
4M, the Alternating Design with a full-sample periodicity of six years and fractional-sample data 
collections three years after each full-sample data collection (“3-6-9” pattern), leads to slightly 
smaller average relative projected absolute errors under the same modeling assumptions than would 
a fixed full-sample periodicity of five years and no fractional-sample data collection at mid-period 
(the “Fixed Design”). Specifically, we have obtained average rel p.a.e. values of 0.016 and 0.014 
for the Alternating Design versus 0.021 and 0.017 fcff the five-year Fixed Design under Models 3A 
and4M respectively. 

The following results were extracted from the summary sheets for Public Schools in Appendix A 
and Appendix B and for Private Schools in Appendix C and Appendix D. 



SASS Schools 




Rel p.a.e. for 50:50 cost ratio 






Fixed Design 


Alternating Design 




Model 3A 


Model 4M 


Model 3A 


Model 4M 


Public 


Iowa 


0.062 


0.047 


0.065 


0.052 


New York 


0.056 


0.042 


0.058 


0.045 


California 


0.043 


0.034 


0.046 


0.037 


National (U.S.) 


0.021 


0.017 


0.016 


0.014 


Private 


National (U.S.) 


0.040 


0.033 


0.039 


0.033 



For Private Schools (national level only), with the same 50:50 cost ratio and use of Models 3A and 
4M, the Alternating Design for a “3-6-9” pattern leads to average rel p.a.e. values of 0.039 and 
0.033 versus 0.040 and 0.033 for the five-year Fixed Design under Modesl 3A and 4M; that is, the 
Alternating Design yields relp.a.e. values that are either the same or lower than the values for 
the Fixed Design for national level Private School estimates. 

Thus, for both Public Schools and Private Schools at the national level, the approximate 
annualized multi-year costs for the Alternating Design and the Fixed Design are equal, but the 
Alternating Design has the advantage of equal or lower average relative projected absolute errors 
and produces updated national estimates every three years instead of every five years. 

Furthermore, the State level estimates for Public Schools are not appreciably higher under the 
Alternating Design (full sample every six years) than under the Fixed Design (full sample every 
five years). 
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5. 



Technical Supplement 



This Technical Supplement provides supplementary information which further describes the 
theoretical and empirical basis for some of the topics and analyses presented in previous sections of 
this report. 



5.1 Equivalent Sample Size Form of the Kalman Filter and Its Application to Random 
Walk Models of Repeated Surveys [cross-reference Sec. 3.2 ] 

Consider a repeated survey system in which the process state is represented by the scalar state 
variable x(j) evolving as a scalar random walk in discrete time, x(j) = x(j-I) + w(j) , where w(j) 
is the process noise term, with scalar survey measurements y(k) given by y(k) = x(k) + b(k) , 
where b(k) is the measurement noise term and the sample size at each survey time k is the scalar 
quantity n(k) and the sample noise variance Bfkj is given by B(k) = R/n(k) , where R is the 
(constant) unit measurement noise variance. The Kalman gain in the optimal filter theorem then 
becomes 

K(k) = C(k\k-T) [ C(k\k-T) + B(k) 

= [C(k-T\k-T) + TQ]/[C(k-T\k-T) + TQ + R/n(k)] , 

which is of the same form as the exponential smoothing parameter in a development due to 
Harrison; see Harrison and Stevens (1976). The error variance equations in the optimal filter 
theorem are now of the form 

Between surveys 

C(k+j\k) = C(k\k)+iQ, 

At surveys 

C(k \k) = [1- K(k) ] C(k I k-T ) , 

where C(0 10), Q , and R are positive scalars and so are K(k ) , C(k I k) , and C(k+j Ik). In 
this development the scalar quantity n°(k I k) was then defined by n°(k I k) = RC‘(k I k) and 
referred to as the “updated equivalent sample size after surveying at survey time k with no 
processing delay.” It was further interpreted in inventory terms as the level of a stock of 
information on hand immediately after ordering n(k) additional units (with no leadtime); that is, 
as an inventory order level. The scalar quantity nfk+j I k) was defined by nfk+j I k) = 
RCfk+j\k) and referred to as the “equivalent sample size remaining at time k+j , j time units 
after the survey time k It was interpreted in inventory terms as the stock on hand j time units 
after ordering and receiving new stock. For a fixed interval T between surveys, assuming the 
system was in steady state, nfk I k-T) was interpreted in inventory terms as the reorder point and 
T as the scheduling period. 

The Kalman gain is then 

K(k) = n(k) / [nfk\k-T) + n(k)] 
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and the updated equivalent sample size is then 



n°(k\k) = nr(k\k-T) + n(k) . 

A further interpretation of n°(k\k) was that it is the size of a survey that would be required to have 
the same degree of precision as that provided by the combined amount n,(k\k-T) + n(k) . This 
development led to an equivalent sample size form of the error variance equations in the optimal 
filter theorem for Scalar Model 1 ; 

Between surveys 

nr(k+j\k) = n°(k\k) [ 1 + jQR-^ n^(k\k) ]-\ 

At surveys 

n°(k I k) = n(k) + n°(k-T 1 k-T) [ 1 + TQR^ n^k-T 1 k-T) . 

S mi th and Barzily (1982) gave a numerical example for a two-item process assumed to be a vector 
random walk with scalar sample size n<j and integer sampling interval T(T= 1,2, ..., 10 years) . 
With assumed cost coefficients for start-up cost and unit costs of interviewing, they demonstrated 
that the cost function J was convex in (Ud, T) and found a minimum for 7 by a numerical 
search. 



5.2 Formulation of Grade-by-Grade Markovian State-Space Model [cross-ref; Sec. 3.3] 

A model set forth in Shumway (1988) was adapted to support the composite modeling and 
estimation of policy variables through the joint use of CCD and SASS data. The theoretical 
formulation as a Markovian state-space model was straightforward. There was a process equation 
(or system equation); 



X(t) 


= 


o(t) 


x(t-l) + 


16x1 




16x16 


16x1 



w(t) 



16x1 



The system state at time zero is x(0) . 

There was an observation equation (or measurement equation); 
y(t) = M(t) x(t) -I- v(t) 

The matrix C>(t) was the “transition matrix.” The matrix M(t) was the “design matrix” or 
“observation matrix.” The vector w(t) was the system noise (or process disturbance) vector; with 
E[w(t)l = 0 . The vector v(t) was the observation noise (or measurement noise) vector; with 
E[v(t)] = 0. 

The process covariance matrix was Q(t) = E[w(t) w(t)’] . 
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The observation covariance matrix was R(t) = E[v(t) v(t)’] . 

The conditional mean was x(t I s) = E[x(t) I y(l), y(2),..., y(s)] . 

The mean-squared error covariance matrix P(t I s) of the estimator x(t I s) 
was P(t I s) = E[(x(t) - x(t I s)) (x(t) - x(t I s))’ I y(l), y(2), ... , y(s)] . 

The calculation of the Kalman filter estimators proceeded by the. forward recursions [cf 
Shumway (1988), p. 177] 

x(tlt-l) = 0(t) x(t-l 1 1-1) 

x(t It) = x(t 1 1-1) + K(t) [y(t) - M(t) x(t 1 1-1)] 

for t=l,...,T and x(0 I 0) = |x. The Kalman gain K(t) was defined as 

K(t) = <D(t) P(t 1 1-1) M(t) [M(t)’ P(t 1 1-1) M(t) + R(t)]-' . 

For M(t) = I , the identity matrix, C>(t) = O, a constant matrix, and R(t) = R, a constant matrix, 
then K(t) reduced to 

K(t) = O P(t 1 1-1) [P(t 1 1-1) + R]-' . 
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Part II Error Projections: Selected Models, Policy Variables, Cost Ratios and Periodicities 

The following four appendices provide documentation for the Schools and Staffing Survey (SASS) 
data used in the present study, for estimates of the pooled mean values of the twelve selected policy 
variables (seven for public schools, four for private schools, and one item applicable to all 
schools), for their estimated standard deviations, and for an average year-to-year magnitude of 
change D for each variable. They also provide dociunentation for the year-to-year evolution of the 
projected absolute errors (p.a.e.) under the selected models, scenarios, and periodicity patterns. As 
noted in the labels for specific tabulation, the errors for each policy variable are typically shown as 
relative projected absolute errors (rel p.a.e.) obtained by dividing the p.a.e. by the estimated mean 
value of the variable. 

For the United States as a whole and for the States of California, Iowa, and New York, Appendix 
A presents the modeling results in terms of projected absolute errors for public schools under 
Models 1, 2 A, 3 A, and 4M with a set of specified scenarios for combinations of periodicity 
choices (intersurvey intervals of 4, 5, or 6 years) and specified cost ratios (variable-to-fixed survey 
cost ratios of 30:70, 40:60, 50:50, 60:40, and 70:30) - that is, for 15 different scenarios. Detailed 
results are shown for each policy variable for national-level estimates and selected summary results 
are shown for the three States as well. These scenarios are referred to as large-sample or full- 
sample scenarios in that the same sample size is assumed to be obtained at each round of data 
collection. The nominal base case scenario is the scenario with a periodicity of 5 years and a 50:50 
cost ratio. The reference expected obtained sample sizes in the tabulations are labeled as “Ref N” 
for the modeling runs at the national level and at the State level for the three States. 

For public schools. Appendix B provides documentation for an alternating large-and-small-sample 
design in which a full data collection is carried out a specified intersurvey interval and a smaller 
data collection is carried out at a mid-point between the two large data collections. This approach 
is illustrated for Models 3A and 4M by a hypothetical design choice designated as the “Interleaved 
3-6-9 Design.” Under this design the same sample sizes that are used under the Appendix A base 
case for a periodicity of 5 years are now used every 6 years for large sample collections and the 
resource savings are assumed to be used to conduct a one-fifth size data collection 3 years after 
each large data collection. Detailed modeling results in terms of projected absolute errors (rel 
p.a.e.) are shown at the national level for the U.S. as a whole and summary results are shown for 
the States of California, Iowa, and New York as well. At the national level, variable-by-variable 
detail is shown for all seven possible cost ratios. The summary sheets display relative p.a.e. 
results for the cost ratios of 30:70, 50:50, and 70:30. 

For private schools. Appendix C provides modeling results (at the national level only) under 
Models 1, 2 A, 3A, and 4M for the four policy variables that are specific to private schools. Again 
all 15 scenarios (periodicities of 4, 5, or 6 years and five cost ratios) were used. 

In Appendix D, the alternating large-and-small-sample design (“Interleaved 3-6-9 Design”) was 
applied to private schools (at the national level only) under Models 3 A and 4M for the four policy 
variables. The national-level summary sheet displays relative p.a.e. results for three of the five 
cost ratios, namely, 30:70, 50:50, and 70:30. 
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Item #1 - Number of students served by chapter 1 services, (public schools) 
(Model 1| (Pooled Mean = 104.591, Estim Std Dev = 150.153, Avg D = 3.855) 
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Item #1 - Number of students served by ehapter 1 services, (public schools) 
IModel 3A| (Pooled Mean = 104.591, Estim Std Dev = 150.153, Avg D = 3.855) 
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Item #4 - Number of K-12 teachers that are new to the school this year, (p 
(Model I| (Pooled Mean = 3.214, Estim Std Dev = 4.000, Avg D = 0.175) 
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Item #6 • Percentage of all schools with minority principals, (public and private schools) 
(Model 1) (Pooled Mean = 0.095, Estim Std Dev = 0.292, Avg D = 0.003) 
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Item U6 - Percentage of all schools with minority principals, (public and private schools) 
IModel 3A| (Pooled Mean = 0.095, Estim Std Dev = 0.292, Avg D = 0.003) 
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Item #7A - Number of students per FTE teacher, (public schools) 
(Model 1| (Pooled Mean = 16.979, Estim Std Dev = 5.280, Avg D = 0.234) 
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Item #7A - Number of students per FTE teacher, (public schools) 
IModel 3A| (Pooled Mean = 16.979, Estim Std Dev = 5.280, Avg D = 0.234) 




r** 

o 



o 

<N 



— <y\oeoe<N r** 

Os — 
fSO 

oooooo oo 



NO 

o 



o 

NO 

ON 



r*. Tf — — . — ir> 
m <N o NO 
O <N m 

oooooo 



es 

>• 

NO 

il 



0U 



o 

o 

00 



o 

rf 

NO 



— rf rr 00 

»«}• <N O NO — 

O <N rn rn ^ ^ 

o o o’ o’ o o 



oo oo 
ON — 
«N O 

o o 



<N 00 

o — 
m o 



U-N (N O ON ON rn NOOO 

Tfro«— NO — NO o — 

OC'it^fn-^Tf mo 

oooooo oo 



o 

00 

NO** 



CNiONt^NONOo m 00 

«r> m — r- (N r- — — 

O fN m m rj* rj* m O 

OOOOOO oo 



o 

o 

NO 



00'^<N — — 
mcNiovo — 
p<NromTf 

o o o o o 



r«* NO 
NO — 
(N O 



NO 

o 



o 

o 

00 

o" 



— r- «r> rf 

TJ-<NONO — 

o <N m ro tT 

o o o o o 



O NO 
r- — 
(N O 

o o’ 



m 

o 

> 

Vi 

il 



o 

o 

o 

On* 



lO — On 00 00 

'^j-movo — 

p (N m m ^ 

o o o’ o’ o 



Tp NO 

r- — 
<N O 



V 

0U 



o 

o 

<N 



o r«* rf m m 

i/> rn — r- <N 

p (N m m rf 

o o o o o 



ON NO 
*-1 
<N O 
O O 



o 

o 



r-rfrM — — 
wi rf (N 00 m 
Orvimmrj- 

o o o o o 



r«* r«* 
00 — 
CM O 



o 

00 

o 



o 

NO 



<N ON NO 
<N O NO 
O <N m m 

d o o o 



VN (N O On 
rt m — NO 
o CNi m m 
d d d d 



NO rf 
m — 
CNI o 
o’ d 



On rf 
m — 
<N p 

d d 



es 

>• 

TJ' 

II 



O 

O 

(N 



o r«* ^ m 
to m — 

O <N m m 

d d d d 



m Tf 
<N O 

d d 



0U 



O 

NO 

r«* 



NO <N O ON 

»r» rf <N r«* 
O <N m m 

d d d d 



On Vi 
rt — 
<N O 

d d 



o 

<N 



Ml AA AA 

NO »n <N 00 
p <N m m 

d d d d 



00 V 

%n — 
o 



^ s I § 



> >• > 
^ V? 'O 



2 . 
oo JT 
> 0> 
< 



es 

> 

Vi 

II 

>v 



es 

> 

II 



TPONo<ntoo» OONO 

m ON NO fs — so — 

p — <N m m ts o 

dddddd dd 



o vo to ON ^No 
moNNOfNr^— so — 

p — C 4 m m 'ti- <N o 

dddddd dd 



IP- — . r«* so NO o 

r}* ON NO (N r- <N 

o — (N m m 

dddddd 



t/> <N 00 r«* NO o 
Tf On NO <N <N 
p — rMmmrf- 

dddddd 



O NO 
r- — 
(N o 

d d 



<N NO 
— 
<N O 

d d 



rM rr ON 00 r- — rr so 

w-N ON NO <N r- <N r- — 

o — (NmmTf <No 

dddddd dd 



00 — r«* NO NO 

m On NO <N r«* 

O — <N m m 

d d d d d 



ON rf 
m — 
<N O 

d d 



— — . r«* so NO 

rf0N\0<NI>- 
O — CN m m 

d d d d d 



v-> <N 00 r«* NO 

Tf On NO <N r«* 

O — <N m m 

d d d d d 



o ^ 
— 
<N O 

d d 



— ^ 
Tf — 
<N O 

d d 



o m On r«* r«* 

V-> On NO <N 

O — CNi m m 

d d d d d 



m rp 
rf — 
(N O 



r- 


to 


o 


On 


00 


NO 




v-> 


£ 


r- 


<N 


r- 






O 




CM 


m 


m 


CM 


o 


d 


d 


d 


d 


d 


d 


d 


CM 


p— 


00 


NO 




r- 


<N 




ON 


NO 


<N 




o 




o 




CM 


m 




CM 


p 


d 


d 


d 


d 




d 


d 



to 00 

rp On NO CM 

o — CM m 
d d d d 



00 <N 
O — 
CM O 

d d 



O m On 
to On NO CM 
O — CM m 

d d d d 



O <N 
CM O 

d d 



CO 



CM 



VO to O 00 
tCN ON CM 

o — <M m 
d d d d 



<M CM 
CM O 



3 



00 CM O 
On m 
O — <M m 

d d d d 



CM O 

d d 






^ ^ ^ ^ ^ 

o *5: a> o> o> 

>:>->->->->- 

to *c "S 5 ■£ 

— CM ^ tr> NO 



«a 0 Jr 

> 4 > 

< 0 ^ 



PageA-9 National (Public) 



Item #8A - Percentage of schools in which various programs and services were available to students, (public schools) 
{Model 1] (Pooled Mean = 0.993, Estim Std Dev = 0.084, Avg D = 0.001) 
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Item #8A - Percentage of schools in which various programs and services were available to students, (public schools) 
(Model 3A| (Pooled Mean = 0.993, Estim Std Dev = 0.084, Avg D = 0.001) 
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Item U9 - Percentage of principals having master's degree, (public schools) 
[Model 1] (Pooled Mean = 0.972, Estim Std Dev = 0.166, Avg D = 0.004) 
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Item #9 - Percentage of principals having master's degree, (public schools) 
(Model 3A| (Pooled Mean = 0.972, Estim Std Dev = 0.166, Avg D = 0.004) 
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Item ^lOA - Percentage of full time teachers who received various types of compensation, (public schools) 
(Model 1| (Pooled Mean = 0.386, Estim Std Dev = 0.486, Avg D = 0.010) 
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Item #11 A - Percentage of full time teachers who were newly hired and who were first time teachers, (public schools) 
IModel 3A| (Pooled Mean = 0.034, Estim Std Dev = 0.181, Avg D = 0.001) 
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Page A-17 National (Public) 



SASS Results, Estimates, and Projected Absolute Errors (California) Public Schools 
Large-Sample Scenarios with Periodicities of 4, 5, or 6 Years 
under Models 1, 2A, 3A, 4M 











oo 


vn 


»n 




On 


CM 


CM ^ 






WP. 










Nn 


o 


CM 


00 


O 




O 


o 


O 


o 




Q 


Os 


cn 


o 




p 


o 


o o 


O 


o 


o 


o 




b£ 


00 


O 


o 


O 


O 


o 


o o 


d 


d 


d 


d 




> 


























< 




























OO 


ON 


o 


"ln~ 


■vb” 


"vn~ 




“ZZ~' 


’Cn*" 




vb 




> 




00 


CM 


NO 


o 


CM 


VO in 


ON 


in 


o 


Mr 


s 




rN 


00 


cn 


CM 


p 




CM CM 




•T 


CM 


CM 




O 






o 


NO 


00 


o 


o o 


d 


d 


d 


d 


on 


TJ 


On 


























rN 






















CA 




























00 




VO 


’vT" 




'""cr” 


"o-'oT 




cn” 


“cn 


in 






m 


On 




NO 


cn 


00 


SJ s 


o 


On 


rr 


NO 


V 


e 


m 




PM 




ppp 


On 


TT 


CM 


o 


o 




C8 




<n 


o 


cn 




O 


o o 


d 


d 


d 


d 


£ 




00 

CM 






CM 




















o 




r«- 


00 


CM 


cn 


VO NO 


WPP 


M- 


NO 


cn 




> 


m 




m 


CM 


ON 


CM 


o ^ 


ON 


in 


o 


VO 






ON 




m 


p 


m 




CM CM 






CM 


CM 




O 


ON 




O 


«n 


o 


O 


o o 


d 


d 


d 


d 




2 


o 

m 




















































NO 






’■«T 


7n 


«n 


VO in 


in 


PPW 


M- 


in 






ON 




cn 


XT 




00 


in m 


o 


ON 


M- 






e 


CM 






»n 


p 


On 


On On 


TT 


CM 


O 


o 




C8 


rS 




o 


cn 




O 


O O 


d 


d 


d 


d 


- 94 




CO 






CM 




















ON 




o 


"oT 


"cm 


CM 


»n o 




CM 




"Jsi 






in 




o 




»n 


»n 


VO oo 


CM 


NO 


CM 


VO 


CS 










cn 


CM 


CO 


CM m 




ON 




Ov 




z 
















CM 




CM* 








CM 




8 


CM 


in 


CM 


in o 


Tf 


CM 


Tf 








«n 




«n 


VO 


in 


vO OO 


CM 




CM 


NO 






m 






m 


CM 


m 


CM m 


ppp 


On 




OV 




« 

z 
















cm" 




cm" 








m 


«n 


M 


00 


m 


o 


o 


o 


NO 


o 


00 




> 


00 


NO 


CM 








On 


ON 


in 


o 


CM 






CM 


00 


rn 


p 


CM 




CM — 




Tf 


CM 


CM 




o 




rn 


o 




00 


o 


o o 


d 


d 


d 


d 




2 


O 

m 
























C/3 




























CM 




"rC” 




"r^ 


"o" 




”cm" 


*in" 


“cm” 


'VT 






ON 


CM 




CM 


in 


00 


CM NO 


o 


On 




in 




e 


CM 


p 


— 


WPP 


m 


ON 


On On 


Mr 


CM 


o 


O 




C8 






o 


cn 


VO* 


o 


O O 


d 


d 


d 


d 


1 




r-« 

CM 






CM 
















o 




«n 


NO 


CM 


in 


o 


NO 


3 8 


m 


00 


m 


"bo 






m 


ON 


00 


00 


in 


ON 


m 


m 


m 


m 


Cs 




— 


CM 


NO 


CM 


cn 


CM 


m m 


p 


00 


p 


00 




z 




























NO 


NO 


CM 


VO 


rr 


NO 


s § 


m 


00 


m 


00 






ON 


ON 


00 


ON 


NO 


ON 


cn 


m 


cn 


cn 






CM 


CM 


VO 


CM 


cn 


CM 


cn m 


p 


00 


p 


00 




|z 




























ri 


m 


M. 


00 


VO 


o 


o 


< 


< 


< 


< 




> 


-•r 




o 


in 


O 




CM — » 


< 










' «* 


o 


p 


cn 




CM 


WPP 


cn m 


Z 




2 


2 


tB 


o 


rn 


o 


«n 


00 


o 


o o 












2 


1^ 

CM 
























35 




























NO 


CM 


o 


cn 


— • 


00 


On 


< 


< 




< 






CM 


NO 


o 


CM 




oo 


00 00 












C5 


Tf 


P 




00 


p 


On 


oo 00 


z 




Z 


2 






ON* 


CM 


o 


CM 




O 


O CD 










OC 

oe 




tn, 

CM 






CM 
















r- 




cn 


"no” 


cn 


NO 


cn 


vO 


m o 


< 




< 


< 


00 




m 


«n 


00 


in 




Nn 


00 










Cs 




CM 


«n 


r-« 


vn 


CM 


in 


CM in 


z 


z 


2 


2 




z 




























NO 


NO 


cn 


NO 


cn 


NO 


cn o 


< 


< 


< 


< 






«n 


«n 


00 


in 




in 


— oo 






< 






« 


«n 


«n 




in 


CM 


in 


CM in 




2 


z 


2 




z 
































i»-s 


























al 




















Cl 










































.3 




.3 






3 

O 


£ £ 


£ 


ja 

3 


*C 

fiu 


ja 

£ 




£ 

W' 


*d 


3 

Op 


a. 

NW> 








bw 








ip< 


kp 


kp 


kp 


kw 










g 








•s c 


o 


01 


01 


ttl 






O 


o 






O 


o 


S '3 


x: 






JC 






O 


o 


‘S 


o 


o 


6 


O g 


o 


o 


o 


O 






JS 


JBS 




JBS 




JC 


JC ^ 


eo 


CO 


CO 


CO 






Cl 


o 


*o 


o 


o 


o 


o ^ 


,01 






01 






c/5 


c/5 


< 


C/5 


c/5 


c/5 


CO < 




H 


H 


H 




E 

Cl 








< 


ffi 


< 


aa 


< 

o 


ffl 

o 


< 


s 








Tj- 


NO 






oe 


00 On 


















% 


% 


% 


% 


% % 


% 






% 




o 

00 



Page A- 18 State of California (Public) Scenarios 



Item #1 - Number of students served by chapter 1 services, (public schools) 
(Pooled Mean = 281.338, Estim Std Dev = 295.278, Avg D = 8.978) 
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Item #4 - Number of K-12 teachers that are new to the school this year, (p 
(Pooled Mean = 3.494, Estim Std Dev = 3.889, Avg D = 0.335) 
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Item #6 - Percentage of all schools with minority principals, (public and private schools) 
(Pooled Mean = 0.1 16, Estim Std Dev = 0.320, Avg D = 0.005) 
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Projected Absolute Errors (State) 

umber of students per FTE teacher, (public schools) 
an = 23.165, Estim Std Dev = 6.265, Avg D = 0.121) 
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Item #8A - Percentage of schools in which various programs and services were available to students, (public schools) 

(Pooled Mean = 0.984, Estim Std Dev = 0.125, Avg D = 0.002) 
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Item U9 - Percentage of principals having master's degree, (public schools) 
(Pooled IVIean = 0.929, Estim Std Dev = 0.256, Avg D = 0.016) 
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Item #9 - Percentage of principals having master's degree, (public schools) 
(Pooled Mean = 0.929, Estim Std Dev = 0.256, Avg D = 0.016) 
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Item #11 A - Percentage of full time teachers who were newly hired and who were first time teachers, (public schools) 

(Pooled Mean = 0.043, Estim Std Dev = 0.203, Avg D = 0.001) 
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Item Ul - Number of students served by chapter 1 services, (public schools) 
(Model 1| (Pooled Mean = 41.240, Estim Std Dev = 31.571, Avg D = 0.951) 
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Item #1 - Number of students served by chapter 1 services, (public schools) 
(Model 3A| (Pooled Mean = 41.240, Estim Std Dev = 31.571, Avg D = 0.951) 
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Item U7\ - Number of students per FTE teacher, (public schools) 
(Model 1| (Pooled Mean = 14.710, Estim Std Dev = 3.530, Avg D = 0.188) 
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Item #7A - Number of students per FTE teacher, (public schools) 
IModel 3A| (Pooled Mean = 14.710, Estim Std Dev = 3.530, Avg D = 0.188) 
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Item #8A - Percentage of schools in which various programs and services were available 

(Pooled Mean = 0.994, Estim Std Dev = 0.080, Avg D = 0.005) 
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Item #8A - Percentage of schools in which various programs and services were available to students, (public schools) 
IModel 3A] (Pooled Mean = 0.994, Estim Std Dev = 0.080, Avg D = 0.005) 



er|c 



o> 

>• 

II 



ooo— > — — oo 

p O O o O O op 

O O O O CD O CD O 



rPOOON — 00^ 

pop — — — oo 

ppoooo oo 

OCDOOOO oo 



mooo — <Nfo oo 

pppppp pp 

o’ooooo oo 



tn ov o rN m oo 

oo — — — — — — 

pppppp pp 

0*0 0 0*0 0 o' o 



NO 

o 

o 



o — (N Tf 
o o o o o 



o o o o o o 



o o 
o o 



^ 00 On — CM 

ooo — — 

p p o p p 

O O O CD O 



0 \ ON 

o o 
o o 



m 00 o — CM 

oo — — — 

o o o p o 

O* CD CD O CD 



Ov 0\ 

s *=» 

O O 
O CD 



^ ON O — CM 

oo— — — 
p p p p o 
o o o o o 



o 

o 

o 



NO O — CM 

OO — — — 

O O O O o 

CD O O O C> 



o o 
o o 
o o 



o CN ro Tf 

o o o o o 

O* CD O CD O 



o o 
o o 



v> 00 o — 
o o — — 

p p p p 
O O O CD 



0 \ 0 \ 
o o 
o o 



VN ON O <N 

o o — — 
o p o o 

O CD O O 



g 



0 \ 
o 
o o 
CD C> 



NO On — CNI 

o o — — 
o o o o 
o o o o 



ON o 

o — 
o p 
o d 



NO O <N 

o — — — 

p p p p 
O* d CD CD 



o o 
o o 
d d 



^ 

o — — — 
o o o o 
d d d d 



o o 
d d 



> 



A 

u 

> 




A 






V 

eo 


> 


> 


> 


> 


sL 

00 

> 


T3 

C 


*s 




JC 

— 




CN 










< 



ed 

D. 



es 

V 

> 

NO 

II 



es 

V 

> 

10 

11 



Tj* 00 On 

oooooo oo 
pppppp oo 
dddddd dd 



■^Nor^ooONON 

oooooo oo 

ppoppo oo 

dddddd dd 



vNNOt^ooONON r«*r- 

oooooo oo 

pppppp 

dddddd dd 



to NO oo On O 
ooooo — 

pppppp 

dddddd 



NO op ON On o 
ooooo — 
oooooo 
dddddd 



00 00 
o o 
o o 
d d 



oo 00 
o o 

p p 

d d 



VO 00 On 

ooooo 
ooooo 
d d d d d 



r- r- 
o o 
o O 
d d 



«rN NO r- 00 On 
OOOOO 
OOOOO 

o d d d d 



i> r- 
O o 
o o 



tCN NO 00 On 

p p p o o 

ooooo 
d d d d d 



r- r- 

§ S 

d d 



p 00 Os 

ooooo 
p o o o o 
d d d d d 



00 00 
o o 
o o 



00 On O 

§ § § S 5 

d d d d d 



00 00 
o o 
o o 



irv so 00 
o o o o 
o o o o 
d d d d 



NO r- 
o o 
o o 



tn NO r" 00 
o o o o 
o o o o 
d d d d 



rs r- 
o o 
o o 
d d 



NO 00 On 

o p o o 
o o o o 
d d d d 



r- 
o o 
o o 
d d 



NO 00 On 

o o o o 
o o p o 
d d d d 



00 00 
o o 
o o 
d d 



oo ^ o 
o o o — 
o o o o 
d d d d 



Urn 

A 






I 3 




d 


V 

>- 


>- 


<L> 

> 


<L> 

> 


> 


du 

oo 

> 


•o 


•o 








c 

CM 








>o 


< 



CO 



00 

00 

QQ 



>- 

Q. 

O 

CD 

OO 

uu 

CQ 



Page A-45 State of Iowa 



A 

e 

o 



Sx3 




U 

On 




"o 

o 

u 


Periodicity = 6 Years 


0.7 


269 


0.005 

0.007 

0.009 

0.011 

0.014 

0.016 

0.010 

0.011 




Periodicity = 6 Years 


0.7 


269 


OOOOOO oo 
oooooo p p 
oooooo oo 


0.6 


230 


0.006 

0.008 

0.010 

0.012 

0.014 

0.016 

0.011 

0.011 


VO 

O 


230 


0.006 

0.002 

0.005 

0.003 

0.005 

0.008 

0.005 

0.005 


0.5 


192 


0.006 

0.008 

0.010 

0.012 

0.014 

0.017 

0.011 

0.012 


0.5 


192 


vb<N«Ar'>‘/%ON 
OOOOOO oo 

oooppp R® 

oooooo oo 


0.4 


154 


r' ON m 'O r- rs rs 

oooooo 
oooooo oo 


o 


154 


<N VO fO VjO ON 

oooooo oo 

ooooop PR 

0 0 0*0 0 0 o o 


0.3 




0.008 

0.010 

0.012 

0.014 

0.016 

0.018 

0.013 

0.013 


0.3 




0.008 

0.002 

0.006 

0.003 

0.006 

0.010 

0.006 

0.006 


Item #9 - Percentage of principals having master's degree, (public s 
(Model 1| (Pooled Mean = 0.988, Estim Std Dev = 0.108, Avg D = 0.002) 
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Item #9 - Percentage of principals having master's degree, (public schools) 
IModel 3A1 (Pooled Mean = 0.988, Estim Std Dev = 0.108, Avg D = 0.002) 
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Item #10A - Percentage of full time teachers who received various types of compensation, (public schools) 
IModel 3A] (Pooled Mean = 0.449, Estim Std Dev = 0.497, Avg D = 0.020) 
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Item //llA • Percentage of full time teachers who were newly hired and who were first tin 
(Model 1| (Pooled Mean = 0.023, Estim Std Dev = 0.150, Avg D = 0.001) 


Periodicity = 5 Years 


G> 


1 1,260 


0.003 

0.005 

0.006 

0.007 

0.008 

0,006 

0.246 


0.6 


o 

00 

o 


0,004 

0.005 

0,006 

0.007 

0.008 

0.006 

0.258 


o 


006 


0.004 

0.005 

0.006 

0.007 

0.009 

0.006 

0.273 


0.4 


720 


0.004 

0.006 

0.007 

0.008 

0.009 

0.007 

0.294 


£0 


540 


0.005 

0,006 

0.007 

0.009 

0.010 

0.007 

0.324 


Periodicity = 4 Years 


o 


00 

o 

o 


0.004 
0.005 
0,006 
I 0.007 

0,005 

0.239 


90 1 


864 


Tp so oo ^ CM 

p O O O O 

O o p p ® 

0 0*0 0 o o 


0.5 


720 


0.004 

0.006 

0.007 

0.008 

0.006 

0.269 


0.4 


576 


0.005 

0.006 

0.007 

0.008 

0,007 

0.292 


0.3 


432 


0.006 

0,007 

0.008 

0.009 

0.007 

0.325 




a 




1st Year 
2nd Year 
3rd Year 
4th Year 
5th Year 
6th Year 

Avg p.a.e, 
Rel p.a.e. 



Periodicity = 6 Years 


o 


1,512 


0.003 

0.001 

0.003 

0.002 

0.003 

0.004 

0.003 

0.116 


VO 

O 


1,296 


1 

0,003 

0.001 

0.003 

0.002 

0.003 

0.005 

0.003 

0.120 


O 


o 

00 

o 


0.004 
1 0.001 
0.003 
0.002 
0.003 
0.005 

0.003 

0.126 


Tp 

o 


864 


Tt — 

OOOOOO om 

ppppop 
OOOOOO o o 


O 


648 


0,005 

0.001 

0.004 

0.002 

0,003 

0.006 

0.003; 

0.147 


Periodicity = 5 Years 


o 


1,260 


0.003 

0.001 

0,003 

0.002 

0.003 

0.002 

0.105 


90 


o 

00 

o 


0.004 

0.001 

0,003 

0.002 

0,003 

0.003 

0.109 


o 


006 j 


1 

i 

0.004 

0.001 

0,003 

0.002 

0.003 

0.003 

0.115 


0.4 


1 720 


0,004 

0.001 

0.003 

0.002 

0.003 

0.003 

0.123 


0.3 


540 


0.005 

0.001 

0,004 

0.002 

0.004 

0.003 

0.134 


Periodicity = 4 Years 


0.7 


00 

o 

o 


0,004 

0.001 

0.003 

0,002 

0.002 

0.105 


1 0.6 


864 


0.004 

0.001 

0.003 

0.002 

0.003i 

0.110 


o 


720 


0.004 

0.001 

0.003 

0.002 

0.003 

0.116 


0.4 


576 


0.005 

0.001 

0,004 

0.002 

0.003 

0.125 


o 


432 


0.006 

0.001 

0.004 

0.002 

0.003 

0.138 




Oi 


Z 


1st Year 
2nd Year 
3rd Year 
4th Year 
5th Year 
6th Year 

Avg p.a.c. 
Rel p.a.e. 






CO 



Page A-50 State of Iowa 



Projected Absolute Errors (State) 



f /3 

"o 

o 

JS 

o 

z 

s 

o. 

« 

JS 

o 

a 

« 

« 

C — 


Periodicity = 6 Years 


1 0.7 


1,512 


0.003 
0.004 
0.004 
0.005 
0.005 
1 0.005 

: 0.004 

0.190 




Periodicity = 6 Years 


0 


1,512 


0.003 
0.003 
0.003 
0.003 
^ 0.004 

0.004 

0.003 

0.148 




1 0.6 1 


VO 

OS 

<N 


0.003 

0.004 

0.005 

0.005 

0.005 

0.005 

; 0.005 

0.200 




0.6 1 


1,296 


0 . 003 ; 

0.003 

0.004 

0.004 

0.004 

0.004 

0.004 

0.158 


146 


1 0.5 i 


O 

oo 

o 


0.004 
0.005 
. 0.005 

j 0.005 

0.005 
0.006 

0.005 

0.214 




1 0.5 1 


0 

00 

0^ 

J 


0.004 

0.004 

0.004 

0.004 

0.004 

0.004 

0.004 

0.170 


UJ 


Tf 

o 


864 


. 0.004 

j 0.005 

0.005 
0.006 
0.006 
0.006 

0.005 

0.233 




Tf 

0 


864 


rt 00 

000000 OOO 

ppoppo p — 

000000 00 


m 


0.3 


648 


0 . 005 ; 

0.006 

0.006 

0.006 

0.007 

0.007 

0.006 

0.260 




0.3 1 


648 


in vn VO vn VO 

000000 0 — 

pppppp 0 <N 

000000 00 


i 

>= 

Ou 

0 

0 

CO 

UJ 

CD 

iO 

tH 


Item #11 A - Percentage of full time teachers who were newly hired and who were first tin 

(Pooled Mean = 0.023, Estim Std Dev = 0.150, Avg D = 0.001] 
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SASS Results, Estimates, and Projected Absolute Errors (New York) Public Schools 
Large-Sample Scenarios with Periodicities of 4, 5, or 6 Years 
under Models 1, 2A, 3A, 4M 
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Item #1 - Number of students served by chapter 1 services, (public schools) 
[Model 1| (Pooled Mean = 13S.356, Estim Std Dev = 184.012, Avg D = 11.003) 
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Item Ul - Number of students served by chapter 1 services, (public schools) 
[Model 3A| (Pooled Mean = 135.356, Estim Std Dev = 184.012, Avg D = 11.003) 
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Projected Absolute Errors (State) 
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Projected Absolute Errors (State) 
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(Pooled Mean = 3.893, Estim Std Dev = 5.170, Avg D = 0.225} 
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Item #7A - Number of students per PTE teacher, (public schools) 
(Model 1| (Pooled Mean = 14.882, Estim Std Dev = 3.981, Avg D = 0.223) 
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Item UIK - Number of students per FTE teacher, (public schools) 
(Model 3A| (Pooled Mean = 14.882, Estim Std Dev = 3.981, Avg D = 0.223) 
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Item #9 - Percentage of principals having master's degree, (public schools) 
[Model 1| (Pooled Mean = 0.977, Estim Std Dev = 0.148, Avg D = 0.004) 
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Item #1 - Number of students served by chapter 1 services, (public schools) 
(Model 3A| (Pooled Mean = 104.591, Estim Std Dev = 150.153, Avg D = 3.855) 
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Projected Absolute Errors (US) — Interleaved 3-6-9 Design 
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Item #6 - Percentage of all schools with minority principals, (public and private schools) 
IModel 3A) (Pooled Mean = 0.095, Estim Std Dev = 0.292, Avg D = 0.003) 
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Item USA - Percentage of schools in which various programs and services were available, (public schools) 
IModel 3A) (Pooled Mean ~ 0.993, Estim Std Dev = 0.084, Avg D “ 0.001) 
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Item #9 - Percentage of principals having master's degree, (public schools) 
[Model 3A| (Pooled Mean = 0.972, Estim Std Dev = 0.166, Avg D = 0.004) 
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Projected Absolute Errors (US) — Interleaved 3-6-9 Design 
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Item U9 - Percentage of principals having master's degree, (public schools) 
IModel 3A| (Pooled Mean = 0.929, Estim Std Dev = 0.256, Avg D = 0.016) 
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Item #10A - Percentage of full time teachers who received various types of compens 
(Pooled Mean = 0.404, Estim Std Dev = 0.491, Avg D = 0.001] 
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Item #6 • Percentage of all schools with minority principals, (public and private schools) 
(Model 3A) (Pooled Mean = 0.017, Estim Std Dev = 0.127, Avg D = 0.005) 
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Item #8A - Percentage of schools in which various programs and services were available, (public schools) 
IModel 3A) (Pooled Mean = 0.994, Estim Std Dev = 0.080, Avg D = 0.005) 
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Item #9 - Percentage of principals having master's degree, (public schools) 
(Model 3A| (Pooled Mean = 0.988, Estim Std Dev = 0.108, Avg D = 0.002) 
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Item #10A - Percentage of full time teachers who received various types of compensation, (public schools) 
(Model 3A| (Pooled Mean = 0.449, Estim Std Dev = 0.497, Avg D = 0.020) 
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Item #8A - Percentage of schools in which various programs and services were available, (public schools) 
(Model 3A| (Pooled Mean = 0.997, Estim Std Dev = 0.052, Avg D = 0.001) 
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Item #9 • Percentage of principals having master's degree, (public schools) 
(Model 3A| (Pooled Mean = 0.977, Estim Std Dev - 0.148, Avg D = 0.004) 
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Item ^lOA - Percentage of full time teachers who received various types of compensation, (public schools) 
[Model 3A| (Pooled Mean = 0,413, Estim Std Dev = 0.492, Avg D = 0.006) 
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Projected Absolute Errors (State) — Interleaved 3-6-9 Design 



LO 

(M 




t/i 

3 

o 

JZ 

o 

Vi 

Z 

3 

O. 

'w' 




o 




0.003 

0.007 

0.009 

0.010 

0.012 

0.013 

0.008 

0.374 


VO 

o 




0.003 
0.007 
0.009 
! 0.010 

1 0.012 

0.013 

0.008 

0.386 


State composi 
"Interleaved 


0.5 




0.003 

0.008 

0.010 

0.011 

0.012 

0.014 

0.009 

0.402 


o 




mooo — Ov CN 

OO— - — — — 0<N 

OOOOOO 

oooooo oo 


0.3 




1 

0.004 j 
0.009 
0,011 
0.012 
0.013 
0.015 

0.010 

0.451 


Item #11 A - % of full time teachers newly hired and who were first time teachei 
(Model 3A| (Pooled Mean = 0.021, Estim Std Dev = 0.144, Avg D = 0.006} 
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Item #7B - Number of 'Students per FTE teacher, (private schools). 
(Pooled Mean = 15.674, Estim Std Dev = 18.494, Avg D = 0.463) 
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Item #7B - Number of students per FTE teacher, (private schools) 
(Pooled Mean = 15.674, Estim Std Dev = 18.494, Avg D = 0.463) 
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Page C-3 National (Private) Scenarios 



Item #8B - Percentage of schools in which various programs and services were available to students, (private schools) 

(Pooled Mean = 0.841) Estim Std Dev = 0.366, Avg D = 0.001) 
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Page C-4 National (Private) Scenarios 



Item #8B - Percentage of schools in which various programs and services were available to students, (private schools) 

(Pooled Mean = 0.841, Estim Std Dev = 0.366, Avg D = 0.001) 



Periodicity = 6 Years 


1 0.7 


4,200 


0.005 

j 0.006 

i 0.006 

1 0.006 

0.007 
0.007 

0.006 

0.007 




Periodicity = 6 Years 


o 


4,200 


1 

0.005 

0.005 

0.005 

0.005 

0.005 

0.005 

0.005 

0.006 


o 


o 

o 


^ ^ so r- r- r- sO oo 

oooooo oo 

OOOOOO oo 

oooooo oo 




VO 

O 


3,600 


0.005 

0.005 

0.005. 

0.005| 

0.005; 

0.005 

0.005 

0.006 


o 


3,000 


0.005 
j 0.006 

, 0.007 

j 0.007 

0.007 
0.008 

0.007 

0.008 




VO 

o 


o 

o 

o^ 

fO 


w^inovovoNO so r- 

oooooo oo 

pppppp oo 

oooooo oo* 


o 


2,400 


>or^r«*ooc©oo o\ 

oooooo oo 

oooooo oo 

oooooo oo 




o’ 


2,400 


vovo^vcvcvp voi^ 

OOOOOO OO 

pppppp po 

oooooo oo 


m 

o 


1,800 


0.007 

0.008 

0.008 

0.009 

0.009 

0.009 

0.008 

0.010 




fO 

o 


o 

o 

00 


0.007 

0.0071 

0.007 

0.007j 

0.007 

0.007 

0.007 

0.008 


Periodicity = 5 Years 


r- 

o 


3,500 


0.005 

0.006 

0.006 

0.007 

0.007 

0.006 

0.007 




Periodicity = 5 Years 


r» 

o 


i 3,500 


0.005 

0.005 

0.005 

0.005 

0.005 

0.005 

0.006 


0.6 


3,000 


0.005 

0.006 

0.007 

0.007 

0.007 

0.007 

0.008 


VO 

o 


3,000 


0.005 

0.005 

0.006 

0.006 

0.006 

0.006 

0.007 


1 0.5 


2,500 


0.006 
} 0.007 

0.007 

1 0.008 

^ 0.008 

0.007 

0.009 


VO 

o 


2,500 


^ so ^ so V© vor» 

o o o o o o o 

p p p p p p p 

o o o o o o o 


1 0.4 


2,000 


0.007 

0.008 

0.008 

0.008 

0.009 

I 0.008| 

0.009 


1 0.4 


2,000 


1 

0.007 

0.007| 

0.007‘ 

0.007] 

0.007 

0.007 

0.008 


o 


o 

o 


0.008 

0.009 

0.009 

0.009 

0.010 

0.009 

0.011 


1 1 


1,500 


600*0 

800*0 

800*0 

800*0 

800*0 

800*0 

800*0 


C 

08 

V 

- 


1 0.7 


2,800 


0.006 

0.007 

0.007 

0.007 

0.007 

0.008 






1 0.7 1 


1 2,800 


0.006 

0.006 

0.006 

0.006 

0.006 

0.007 


90 


2,400 


VO oo r«. oo 

O O Q O O O 

poop op 

o o o o o o 


S 

08 


r 90 


2,400 


0.006 

0.006 

0.006 

0.006 

0.006 

0.007 


II 

>v 

*3 


o 


2,000 


0.007 

0.008 

0.008 

0.008 

0.008 

0.009 


n 

[3 


VO 

o 


1 2,000 


0.007! 

0.007j 

0.007! 

0.007 

0.007 

0.008 


*£ 

fiu 


o 


1,600 


010*0 

‘sooo 

600*0 

600*0 

800*0 

400*0 


O ' 
'u 

a. 


0.4 


1,600 


0.007 

0.007 

0.007 

0.008 

0.007 

0.009 




0.3 


1,200 


0.008 

0.010 

0.010; 

0.010 

o.oio 

0.011 




0.3 


1,200 


010*0 

600*0 

600*0 

600*0 

600*0 

800*0 




a ; 


z: 


1st Year 
2nd Year 
3rd Year 
4th Year 
5th Year 
6th Year 

Avg p.a.e. 
Rel p.a.e. 


(Model 4M| 




Qi 


z: 


1st Year 
2nd Year 
3rd Year 
4th Year 
5th Year 
6th Year 

Avg p.ae. 
Rel p.a.e. 



CD 

CO 

cv» 



CO 




Page C-5 National (Private) Scenarios 



Item #10B - Percentage of full time teachers who received various types of compensation, (private schools) 
(Pooled Mean = 0.269, Estim Std Dev = 0.442, Avg D = 0.016) 
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Page C>6 National (Private) Scenarios 



Item #10B - Percentage of full time teachers who received various types of compensation, (private schools) 
(Pooled Mean = 0.269, Estim Std Dev = 0.442, Avg D = 0.016) 
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Page C-7 National (Private) Scenarios 



Projected Absolute Errors (US) 
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Item #11B - Percentage of full time teachers who were newly hired and who were first time teachers, (private teachers) 

(Pooled Mean = 0.06S, Estim Std Dev = 0.246, Avg D = 0.004) 
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Page C-8 National (Private) Scenarios 



Projected Absolute Errors (US) 
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Item #7B - Number of students per FTE teacher, (private schools) 
(Model 3A| (Pooled Mean = 15.674, Estim Std Dev = 18.494, Avg D = 0.463) 
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Item U\OB - Percentage of full time teachers who received various types of compensation, (private schools) 
(Model 3A| (Pooled Mean = 0.269, Estim Std Dev = 0.442, Avg D - 0.016) 
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Projected Absolute Errors (US) — Interleaved 3-6-9 Design 
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comments on School Reform, Governance, and Finance 


Dan Kasprzyk 


96-12 (June) 


Predictors of Retention, Transfer, and Attrition of 
Special and General Education Teachers: Data from the 
1989 Teacher Followup Survey 


Dan Kasprzyk 


96-13 (June) 


Estimation of Response Bias in the NHES:95 Adult 
Education Survey 


Steven Kaufinan 


96-14 (June) 


The 1995 National Household Education Survey: 
Reinterview Results for the Adult Education 
Component 


Steven Kaufinan 
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96-15 (June) 


Nested Structures: District-Level Data in the Schools 
and Staffing Survey 


Dan Kasprzyk 


96-16 (June) 


Strategies for Collecting Finance Data from Private 
Schools 


Stephen 

Broughman 


96-17 (July) 


National Postsecondary Student Aid Study: 1996 Field 
Test Methodology Report 


Andrew G. 
Malizio 


96-18 (Aug.) 


Assessment of Social Competence, Adaptive 
Behaviors, and Approaches to Learning with Young 
Children 


Jerry West 


96-19 (Oct.) 


Assessment and Analysis of School-Level 
Expenditures 


William Fowler 


96-20 (Oct.) 


1991 National Household Education Survey 
(NHES:91) Questionnaires: Screener, Early Childhood 
Education, and Adult Education 


Kathryn Chandler 


96-21 (Oct.) 


1993 National Household Education Survey 
(NHES:93) Questionnaires: Screener, School 
Readiness, and School Safety and Discipline 


Kathryn Chandler 


96-22 (Oct.) 


1995 National Household Education Survey 
(NHES:95) Questionnaires: Screener, Early Childhood 
Program Participation, and Adult Education 


Kathryn Chandler 


96-23 (Oct.) 


Linking Student Data to SASS: Why, When, How 


Dan Kasprzyk 


96-24 (Oct.) 


National Assessments of Teacher Quality 


Dan Kasprzyk 


96-25 (Oct.) 


Measures of Inservice Professional Development: 
Suggested Items for the 1998-1999 Schools and 
Staffing Survey 


Dan Kasprzyk 


96-26 (Nov.) 


Improving the Coverage of Private Elementary- 
Secondary Schools 


Steven Kaufrnan 


96-27 (Nov.) 


Intersurvey Consistency in NCES Private School 
Surveys for 1993-94 


Steven Kaufrnan 
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96-28 (Nov.) 


Student Learning, Teaching Quality, and Professional 
Development: Theoretical Linkages, Current 
Measurement, and Recommendations for Future Data 
Collection 


Mary RoIIefson 


96-29 (Nov.) 


Undercoverage Bias in Estimates of Characteristics of 
Adults and 0- to 2- Year-Olds in the 1995 National 
Household Education Survey (NECES:95) 


Kathryn Chandler 


96-30 (Dec.) 


Comparison of Estimates from the 1995 National 
Household Education Survey (NHES:95) 


Kathryn Chandler 


97-01 (Feb.) 


Selected Papers on Education Surveys: Papers 
Presented at the 1996 Meeting of the American 
Statistical Association 


Dan Kasprzyk 


97-02 (Feb.) 


Telephone Coverage Bias and Recorded Interviews in 
the 1993 National Household Education Survey 
(NHES:93) 


Kathryn Chandler 


97-03 (Feb.) 


1991 and 1995 National Household Education Survey 
Questionnaires: NHES:91 Screener, NHES:91 Adult 
Education, NHES:95 Basic Screener, andNHES:95 
Adult Education 


Kathryn Chandler 


97-04 (Feb.) 


Design, Data Collection, Monitoring, Interview 
Administration Time, and Data Editing in the 1993 
National Household Education Survey (NHES:93) 


Kathryn Chandler 


97-05 (Feb.) 


Unit and Item Response, Weighting, and Imputation 
Procedures in the 1993 National Household Education 
Survey (NHES:93) 


Kathryn Chandler 


97-06 (Feb.) 


Unit and Item Response, Weighting, and Imputation 
Procedures in the 1995 National Household Education 
Survey (NHES:95) 


Kathryn Chandler 


97-07 (Mar.) 


The Determinants of Per-Pupil Expenditures in Private 
Elementary and Secondary Schools: An Exploratory 
Analysis 


Stephen 

Broughman 


97-08 (Mar.) 


Design, Data Collection, Interview Timing, and Data 
Editing in the 1995 National Household Education 
Survey 


Kathryn Chandler 
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97-09 (Apr.) 


Status of Data on Crime and Violence in Schools: Final 
Report 


Lee Hoffman 


97-10 (Apr.) 


Report of Cognitive Research on the Public and Private 
School Teacher Questionnaires for the Schools and 
Staffing Survey 1993-94 School Year 


Dan Kasprzyk 


97-11 (Apr.) 


International Comparisons of Inservice Professional 
Development 


Dan Kasprzyk 


97-12 (Apr.) 


Measuring School Reform: Recommendations for 
Future SASS Data Collection 


Mary Rollefson 


97-13 (Apr.) 


Improving Data Quality in NCES: Database-to-Report 
Process 


Susan Ahmed 


97-14 (Apr.) 


Optimal Choice of Periodicities for the Schools and 
Staffing Survey: Modeling and Analysis 


Steven Kaufinan 
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